Presenting Building Information Using Video And Building Models

ABSTRACT

Techniques are described for using computing devices to perform automated operations related to providing visual information of multiple types in an integrated manner about a building or other defined area. The techniques may include generating and presenting a GUI (graphical user interface) on a client device that includes a computer model of the building’s interior with one or more first types of information (e.g., in a first pane of the GUI), and simultaneously presenting other types of related information about the building interior (e.g., in additional separate GUI pane(s)) that is coordinated with the first type(s) of information being currently displayed. The computer model may be a 3D (three-dimensional) or 2.5D representation generated after the house is built and showing the actual house’s interior (e.g., walls, furniture, etc.), and may be displayed to a user of a client computing device in a displayed GUI with various user-selectable controls.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation-in-part of co-pending U.S.Non-Provisional Pat. Application No. 17/589,842, filed Jan. 31, 2022 andentitled “Presenting Building Information Using Building Models”, whichis a continuation-in-part of U.S. Non-Provisional Patent Application No.17/064,608, filed Oct. 7, 2020 and entitled “Presenting IntegratedBuilding Information Using Building Models” (now U.S. Pat. No.11,238,652), which is a continuation of U.S. Non-Provisional Pat.Application No. 16/681,787, filed Nov. 12, 2019 and entitled “PresentingIntegrated Building Information Using Three-Dimensional Building Models”(now U.S. Pat. No. 10,825,247), each of which is hereby incorporated byreference in their entirety. This application is also acontinuation-in-part of co-pending U.S. Non-Provisional Pat. ApplicationNo. 18/099,892, filed Jan. 20, 2023 and entitled “Multi-Party VirtualShowing System With Configurable Independent Communication Channels AndData”, which is a continuation of U.S. Non-Provisional Pat. ApplicationNo. 16/852,948, filed Apr. 20, 2020 and entitled “Multi-Party VirtualShowing System With Configurable Independent Communication Channels AndData” (now U.S. Pat. No. 11,574,373), which is hereby incorporated byreference in its entirety.

TECHNICAL FIELD

The following disclosure relates generally to techniques for providingvisual information of multiple types about a defined area, such as topresent a video of an interior of an as-built building along with acomputer model of the building interior and/or multiple other types ofinformation about the building interior in an integrated manner (e.g.,in a simultaneous and coordinated manner).

BACKGROUND

In various fields and circumstances, such as architectural analysis,property inspection, real estate acquisition and development, remodelingand improvement services, general contracting and other circumstances,it may be desirable to view information about the interior of a house,office, or other building without having to physically travel to andenter the building, including to determine actual as-built informationabout the building rather than design information from before thebuilding is constructed. However, it can be difficult or impossible toeffectively display visual information about building interiors to usersat remote locations, such as to enable a user to fully understand thelayout and other details of the interior, including under varyingconditions.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram depicting an exemplary building interior environmentand computing system(s) for use in embodiments of the presentdisclosure, including to generate and present information representingthe building interior.

FIGS. 2A-2X illustrate examples of automated operations for presentingvisual information that includes a 3D (three-dimensional) computer modelof a building’s interior along with multiple additional types ofinformation about the building interior in a simultaneous andcoordinated manner.

FIG. 3 is a block diagram illustrating computing systems suitable forexecuting embodiments of one or more systems that perform at least someof the techniques described in the present disclosure.

FIG. 4 illustrates an example embodiment of a flow diagram for an ImageCapture and Analysis (ICA) system routine in accordance with anembodiment of the present disclosure.

FIGS. 5A-5B illustrate an example embodiment of a flow diagram for aFloor Map Generation Manager (FMGM) system routine in accordance with anembodiment of the present disclosure.

FIG. 6 illustrates an example embodiment of a flow diagram for aBuilding Information Integrated Presentation (BIIP) system routine inaccordance with an embodiment of the present disclosure.

DETAILED DESCRIPTION

The present disclosure describes techniques for using one or morecomputing devices to perform automated operations related to providingvisual information of multiple types about a defined area, such asinformation about a building that is presented in an integrated mannerand by using a computer model of the building’s interior. In at leastsome embodiments, the techniques include generating and presenting a GUI(graphical user interface) on a client device that includes a visualrepresentation of a computer model of the building’s interior with oneor more first types of information (e.g., in a first pane of the GUI),and to simultaneously present other types of related information aboutthe building interior (e.g., in one or more additional separate panes ofthe GUI) that is coordinated with the first type(s) of information beingcurrently displayed. The building may, for example, be a house, thecomputer model of the building’s interior may be a 3D(three-dimensional) or 2.5D (two and a half dimensional) or 2D(two-dimensional) floor plan representation that is generated after thehouse is built and that shows the house’s actual interior (e.g., typesof building information such as walls; windows; doors; stairs;fireplaces; kitchen islands; cabinets; counters; lighting and/orplumbing fixtures and associated built-in elements such as sinks andshowers/baths; built-in and/or moveable appliances, such as ovens,stoves, trash compactors, water heaters, HVAC equipment, refrigerators,washers, dryers, etc.; types of infrastructure, such as using wiringand/or piping, including electrical outlets, cable outlets, telephoneoutlets, air vents, sprinklers, speaker locations, security cameras,etc.; curtains and other furnishings; materials used for surfaces suchas kitchen countertops, wallpaper, paint, floor coverings, etc.;furniture; fire extinguishers; etc.), and the types of presentedinformation about the building interior may include visualizations ofthe computer model (e.g., 3D and/or 2D and/or 2.5D), panorama images(e.g., 360° panorama images with 360° of coverage around a verticalaxis), 2D (two-dimensional) perspective photos and other images, videos,an interactive tour of inter-connected viewing/capture locations, andvarious other types of information. Additional details are includedbelow regarding the automated operations of the computing device(s)involved in the generating and presenting of the various types ofcoordinated information about a building’s interior, and some or all ofthe techniques described herein may, in at least some embodiments, beperformed at least in part via automated operations of a BuildingInformation Integrated Presentation (“BIIP”) system, as discussedfurther below.

The types of information that are presented about a building’s interior(and in some cases surroundings) may have various forms in variousembodiments, and may be acquired in various manners. In addition, atleast some of those types of information may be associated withcorresponding positions in a computer model of the building (e.g., a 3Dmodel with full height information represented; a 2.5D model withpartial representations of height represented; a 2D floor map modelwithout height represented, such as using an orthographic top view orother overhead schematic view; etc.). As one example, types ofadditional information about a building may include pieces of data suchas one or more of the following, and may be associated with locationsfrom which the information was captured and/or locations that are shownor otherwise represented in the captured information: photos or otherimages (e.g., 2D perspective images, 360 panorama images and/or otherpanorama images, etc.), such as to be associated with viewing locations(also referred to at times as ‘capture locations’ or ‘recordinglocations’ or ‘viewing/capture locations’) within the rooms of thebuilding where they were taken; videos, such as to each include asequence of multiple images and optionally associated audio and tooptionally be associated with viewing locations within the rooms of thebuilding where it is taken (e.g., as part of a ‘virtual showing’ tourvideo of the building in which a device at the building, sometimesreferred to as a ‘presenter device’, records video of some or all of theinside and/or outside of the building, such as to provide a tour of thebuilding, and transmits or otherwise provides that video over one ormore computer networks to one or more other remote client devices,whether in a near-real-time or real-time manner, such as withinmilliseconds or seconds or minutes of the recording, or at a later timeas part of playback of the video - when the video is transmitted in anear-real-time or real-time manner, it may further be part of aninteractive communication session in which additional communicationstake place between the one or more client devices and optionally thepresenter device, such as textual and/or audio communications, and suchan interactive communication session may similarly take place during aplayback of a prerecorded video between multiple devices receiving theplayback); textual and/or audio annotations or other descriptions ofparticular points of interest (POIs) in the building’s rooms or otherlocations, including types of building information automaticallyidentified via analysis of visual data from acquired images for thebuilding; other audio information, such as recordings of ambient noisethat are associated with recording locations within the rooms of thebuilding where they were taken; a time-lapse or other accelerated videoand/or animation of an area in or around a home, such as from a frontdoor and/or front window of a house to show traffic or other informationover a first period of time (e.g., 1 hour, 4 hours, 8 hours, 12 hours,24 hours, etc.) that is presented over a shorter period of time (e.g.,10 seconds, 30 seconds, 1 minute, 5 minutes, 10 minutes, etc.); otherexternal information from an environment surrounding the building, suchas about nearby buildings and/or vegetation (e.g., as identified fromsatellite or other external images of the building, such as taken fromthe building and/or from a nearby street or yard; as identified fromdata in one or more external databases or other information sources,such as street maps or other government records; etc.), whether bydisplaying the actual images or by generating and displaying visualrepresentations of particular external elements that are identified andmodeled from images or other external information sources; lightinginformation for an interior of a building and/or its surroundings, suchas simulated lighting for one or more rooms of a building (e.g., tosimulate daylight entering the one or more rooms at one or more definedtimes, to show actual lighting from interior lights in the one or morerooms at one or more defined times; etc.); in-room images for a roomthat are projected on the walls of the room shown in the model;user-generated and/or ‘crowd-sourced’ information provided by one ormore end users about the building, such as interior and/or exteriorimages, descriptions, questions, answers, etc. that are associated withparticular rooms or locations within rooms; estimated scale informationsuch as room width, length and/or height dimensions; geographicallocation and/or orientation information for the building; a 2D(two-dimensional) floor map of the building interior, such as using anoverhead schematic view (e.g., an orthographic top view); etc.Additional details are included elsewhere herein regarding types ofinformation that may be presented about a building for a house or otherbuilding in a displayed GUI, such as in a coordinated and simultaneousmanner and/or based at least in part on a generated computer model ofthe building.

In addition, the automated operations may include controlling howmultiple types of information are presented in various embodiments abouta building’s interior (and in some cases, the building’s surroundings),such as in a simultaneous and coordinated manner, a sequential manner(e.g., as selected by a user), etc. In at least some embodiments, suchautomated operations include presenting a GUI with multiple panes thatare simultaneously displayed, and with each pane including informationof a different type that are all related to a common location or area inthe building interior or other common aspect/feature of the buildinginterior. As one non-exclusive example, a first pane may be displaying aphoto taken in a room of the building to show at least some of thatroom, a second pane may be displaying a portion of a 3D computer modelof the building that includes the room, and a third pane may bedisplaying a video taken within the room or a part of an interactivetour of the building that includes one or more viewing locationssituated within the room - in addition, in at least some embodiments,one of the panes may be a primary pane (e.g., that is larger than othersecondary panes, and/or that has user-selectable controls via which theuser may interact with the content shown in that pane in variousmanners), and the GUI may further enable the user to easily switchcontent between the primary pane and one of the secondary panes, or tootherwise change information shown in one or more of the panes. In otherembodiments and/or situations, only a single pane may be displayed, inwhich multiple types of information about a building may be sequentiallydisplayed, such as in response to selections by a user.

Various user-selectable controls may also be displayed in or otherwiseassociated with one or more of the displayed panes (e.g., with theprimary pane, and optionally in a contextual manner based on the type ofcontent displayed in the primary pane) and provide functionality to makevarious types of modifications to the displayed information, such as oneor more of the following: to change the contents of a secondary pane toa primary pane; to change the contents of the primary pane or asecondary pane to another type of information that is not currentlydisplayed, with the new information of the other type similarly beingcoordinated with the content currently displayed in the other panes; tochange the contents of a single displayed pane with at least one type ofinformation about a building, such as to another type of information forthe building that is not currently displayed, to information aboutanother building, etc.; to toggle on and off one or more additionaltypes of information that are overlaid on the contents of a pane, suchas textual and/or audio descriptions, information about points ofinterest (including types of building information automaticallyidentified via analysis of visual data from acquired images for thebuilding), questions and/or answers, lighting information, etc.; to adduser-specified content (e.g., a photo, a description, a question, etc.)to the content of a particular pane and/or to a location of the buildingshown in that pane; etc. Additional details are included elsewhereherein regarding the presentation of one or more types of informationabout a building, including about types of user-selectable controls andother user selections in a displayed GUI, such as for multiple types ofinformation about a building that are displayed in a simultaneous andcoordinated manner.

The described techniques provide various benefits in variousembodiments, including to use 3D models and/or 2.5D models and/or 2Dfloor map models of multi-room buildings and other structures (e.g.,that are generated from images acquired in the buildings or otherstructures) to display various types of information about buildinginteriors, such as in a coordinated and simultaneous manner with othertypes of related information, including to use information about theactual as-built buildings (e.g., internal structural components and/orother interior elements, nearby external buildings and/or vegetation,actual building geographical location and/or orientation, actual typicalweather patterns, etc.) rather than using information from plans on howthe building is designed and should theoretically be constructed. Suchdescribed techniques may further provide benefits in at least someembodiments for allowing improved automated navigation of a building bymobile devices (e.g., semi-autonomous or fully-autonomous vehicles) viause of information of various types, including to significantly reducetheir computing power used and time used to attempt to otherwise learn abuilding’s layout. In addition, in some embodiments the describedtechniques may be used to provide an improved GUI in which an end usermay more accurately and quickly obtain information about a building’sinterior (e.g., for use in navigating that interior, such as via avirtual interactive tour), including in response to search requests, aspart of providing personalized information to the end user, as part ofproviding value estimates and/or other information about a building toan end user, etc. Various other benefits are also provided by thedescribed techniques, some of which are further described elsewhereherein.

For illustrative purposes, some embodiments are described below in whichspecific types of information are acquired, generated, used and/orpresented in specific ways for specific types of structures and by usingspecific types of devices - however, it will be understood that thedescribed techniques may be used in other manners in other embodiments,and that the invention is thus not limited to the exemplary detailsprovided. As one non-exclusive example, while various types ofinformation related to house interiors may be used in some situations,it will be appreciated that one or more of such information types may besimilarly used in other embodiments for other types of buildings (orother structures or layouts) separate from houses and/or for other partsof a house or other building (e.g., for external walls; surroundingyards; surrounding supplemental structures, such as a garage, shed,barn, etc.; roofs; etc.). As another example, while various types ofinformation for models of houses or other buildings may be used fordisplay to assist viewers in navigating the buildings or otherwiseunderstanding the buildings’ interiors, at least some such types ofinformation may be used in other manners in other embodiments. Inaddition, the term “building” refers herein to any partially or fullyenclosed structure, typically but not necessarily encompassing one ormore rooms that visually or otherwise divide the interior space of thestructure - non-limiting examples of such buildings include houses,apartment buildings or individual apartments therein, condominiums,office buildings, commercial buildings or other wholesale and retailstructures (e.g., shopping malls, department stores, warehouses, etc.),etc. The term “acquire” or “capture” as used herein with reference to abuilding interior, viewing location, or other location (unless contextclearly indicates otherwise) may refer to any recording, storage, orlogging of media, sensor data, and/or other information related tospatial and/or visual characteristics of the building interior orsubsets thereof, such as by a recording device or by another device thatreceives information from the recording device. In addition, variousdetails are provided in the drawings and text for exemplary purposes,but are not intended to limit the scope of the invention. For example,sizes and relative positions of elements in the drawings are notnecessarily drawn to scale, with some details omitted and/or providedwith greater prominence (e.g., via size and positioning) to enhancelegibility and/or clarity. Furthermore, identical reference numbers maybe used in the drawings to identify similar elements or acts.

FIG. 1 is an example block diagram of various computing devices andsystems that may participate in the described techniques in someembodiments. In particular, an Interior Capture and Analysis (“ICA”)system (e.g., a system 160 that is executing on one or more servercomputing systems 180, and/or a system provided by application 155executing on one or more mobile image acquisition devices 185) hasacquired images 165 (e.g., 360° spherical panorama images inequirectangular format), such as with respect to one or more buildingsor other structures (not shown in FIG. 1 ), and a Floor Map GenerationManager (“FMGM”) system (e.g., a system 160 that is executing on one ormore server computing systems 180, and/or a system provided byapplication 157 executing on one or more mobile image acquisitiondevices 185) has used the acquired images 165 and optionally otherinformation to generate one or more 2D floor maps 165 and/or computermodels 165 (e.g., 3D and/or 2.5D models) for the one or more buildingsor other structures. FIG. 1 shows one example of acquisition of panoramaimages for a particular house at multiple capture/viewing locations 210(e.g., 210A-210J) within the house interior and exterior (e.g., withlocation 2101 on an external area 118, such as a deck or patio, andlocation 210J in another external area, such as a patio or backyard orgarden), and FIGS. 2A-2X illustrate additional details about using acomputer model generated from such panorama images to control howmultiple types of information are presented about the house, such as ina simultaneous and coordinated manner, and as discussed further below.

A BIIP (Building Information Integrated Presentation) system 140 isfurther executing on one or more server computing systems to usebuilding models, maps and images 145 (e.g., acquired from information165) and/or other mappingrelated information or associated information(not shown) in order to control the presentation of multiple types ofinformation (including such building models 145) about the house orother building, such as in a simultaneous and coordinated manner. Aspart of doing so, the BIIP system may receive requests or instructionsor other information via computer network(s) 170 from end users ofbuilding information viewer client computing devices 175 about types ofinformation to include, before generating and providing such informationfor display on the client computing devices 175, and may furtheroptionally obtain and use supporting information in some embodimentssupplied by BIIP system operator users via computing devices 105 andintervening computer network(s) 170 to configure or modify operations ofthe BIIP system. Additional details related to the automated operationof the BIIP system are included elsewhere herein, including with respectto FIGS. 2A-2X and FIG. 6 . In some embodiments, the ICA system(s) 160and/or FMGM system(s) 160 and/or BIIP system 140 may execute on the sameserver computing system(s), such as if two or more of the systems areoperated by a single entity or are otherwise executed in coordinationwith each other (e.g., with some or all functionality of such systemsintegrated together into a larger system), while in other embodimentsthe BIIP system may instead operate separately from such an ICA system(e.g., without an ICA system by instead obtaining panorama images orother images from one or more external sources) and/or may insteadoperate separately from such an FMGM system (e.g., without an FMGMsystem by instead obtaining 2D floor maps or other computer models ofbuildings from one or more external sources).

Various components of the mobile image acquisition device 185 areillustrated in FIG. 1 , including a browser 162 and/or an ICA systemapplication 155 and/or an FMGM system application 157 that are executedin memory 152 of the device 185 by one or more hardware processors 132,and including one or more imaging systems 135 to acquire visual data.The illustrated embodiment of mobile device 185 further includes one ormore sensor modules 148 that include a gyroscope 148 a, accelerometer148 b and compass 148 c in this example (e.g., as part of one or moreIMU units, not shown separately, on the mobile device), optionally a GPS(or Global Positioning System) sensor 137 or other positiondetermination sensor (not shown in this example), a display system 142,a control system 147 to control acquisition of images (e.g., to rotatethe mobile device), etc. - while not illustrated in this example, insome embodiments the device 185 may further have or be associated withone or more depth sensors or other distance-sensing devices that providedepth data to surrounding objects, while in other embodiments the device185 does not have any such associated depth sensors or otherdistance-sensing devices. Other computing devices/systems 105, 175 and180 may include various hardware components and stored information in amanner analogous to mobile device 185, but are not shown in this examplefor the sake of brevity, although some related further details arediscussed below with respect to FIG. 3 .

In the example of FIG. 1 , the ICA system may perform automatedoperations involved in acquiring multiple images at multiple associatedviewing locations (e.g., in multiple rooms or other locations within abuilding or other structure and optionally around some or all of theexterior of the building or other structure), such as using visual dataacquired via the mobile device(s) 185, and for subsequent use ingenerating and providing a representation of an interior of the buildingor other structure. For example, in at least some such embodiments, suchtechniques may include using one or more mobile devices (e.g., a camerahaving one or more fisheye lenses or other lenses that simultaneouslycapture 360° horizontally around a vertical axis, such as to produce a360° spherical panorama image without rotation; a camera having one ormore fisheye lenses or other lenses that capture less than 360°horizontally at a given time, and that is mounted on a rotatable tripodor otherwise having an automated rotation mechanism to enable 360° imagecapture over a period of time corresponding to the rotation; a smartphone held and moved by a user, such as to rotate the user’s body andheld smart phone in a 360° circle around a vertical axis; a camera heldby or mounted on a user or the user’s clothing; a camera mounted on anaerial and/or ground-based drone or robotic device; etc.) to capturedata from a sequence of multiple viewing locations within multiple roomsof a house (or other building), and to optionally further capture datainvolved in movement or travel between some or all of the viewinglocations for use in linking the multiple viewing locations together,but without having distances between the viewing locations beingmeasured or having other measured depth information to objects in anenvironment around the viewing locations (e.g., without using anydepth-sensing sensors). After a viewing location’s information iscaptured, the techniques may include producing a panorama image fromthat viewing location (e.g., a 360° panorama image that includes 360° ofhorizontal coverage around a vertical axis, a 360° spherical panoramaimage that further shows the surrounding room in an equirectangularformat, another type of panorama image in another format, etc.), andthen providing the panorama images for subsequent use by an FMGM systemand/or BIIP system. Additional details related to embodiments of asystem providing at least some such functionality of an ICA system areincluded in U.S. Non-Provisional Pat. Application No. 16/236,187, filedDec. 28, 2018 and entitled “Automated Control Of Image Acquisition ViaUse Of Acquisition Device Sensors”; in U.S. Non-Provisional Pat.Application No. 16/190,162, filed Nov. 14, 2018 and entitled “AutomatedMapping Information Generation From Inter-Connected Images”; and in U.S.Non-Provisional Pat. Application No. 15/649,434, filed Jul. 13, 2017 andentitled “Connecting And Using Building Interior Data Acquired FromMobile Devices” (which includes disclosure of a BICA system that is oneexample embodiment of an ICA system generally directed to obtaining andusing panorama images from within one or more buildings or otherstructures); each of which is incorporated herein by reference in itsentirety.

In the example of FIG. 1 , the FMGM system may perform automatedoperations involved in using images acquired at multiple associatedviewing locations (e.g., in multiple rooms or other locations within abuilding or other structure and optionally around some or all of theexterior of the building or other structure) to generate a 2D floor mapfor the building or other structure and/or to otherwise generate acomputer model for the building or other structure (e.g., a 3D modeland/or a 2.5D model), such as by analyzing visual information availablein the images, and for providing a representation of an interior of thebuilding or other structure (e.g., for subsequent use in controlling thesimultaneous and coordinated presentation of multiple types ofinformation about the interior of the building or other structure). Forexample, in at least some such embodiments, such techniques may includeanalyzing one or more images taken in a room to determine a shape of theroom and/or to identify inter-room passages (e.g., doorways and otheropenings in walls) into and/or out of the room. After the shapes of someor all of the rooms are determined, the techniques may further includepositioning the room shapes relative to each other to form a 2D floormap (e.g., based at least in part on connecting inter-room passagesbetween rooms, and optionally using travel or other movement informationcaptured between the viewing locations to determine relative locationsof the viewing locations with respect to each other), optionallycombined with height and/or other size information to generate a 3Dand/or 2.5D model of the building or other structure, and then providingthe generated computer model(s) and optionally 2D floor map forsubsequent use by the BIIP system. Additional details related toembodiments of a system providing at least some such functionality of anFMGM system are included in U.S. Non-Provisional Pat. Application No.16/190,162, filed Nov. 14, 2018 and entitled “Automated MappingInformation Generation From Inter-Connected Images”; and in U.S.Provisional Pat. Application No. 62/893,108, filed Aug. 28, 2019 andentitled “Automated Tools For Generating Mapping Information ForBuildings” (which includes disclosure of an MIGM system that is oneexample embodiment of a FMGM system generally directed to generatingfloor maps and other computer models for one or more buildings or otherstructures based in part of input from one or more system operatorusers); each of which is incorporated herein by reference in itsentirety.

One or more end users (not shown) of one or more building informationviewer client computing devices 175 may each further interact overcomputer networks 170 with the BIIP system 140 (and optionally the ICAsystem 160 and/or FMGM system 160), such as to obtain, display andinteract with a generated 3D computer model and/or other model (e.g., a2D floor map) that are part of multiple types of information about thehouse or other building being presented in a GUI displayed on a device175 (e.g., based at least in part on user-specified conditions), such asin a simultaneous and coordinated manner. In addition, while notillustrated in FIG. 1 , a 3D computer model (or portion of it) and/orother model (e.g., 2D floor map) may be linked to or otherwiseassociated with one or more additional types of information, such as oneor more associated and linked images or other associated and linkedinformation, including for a two-dimensional (“2D”) floor map of abuilding to be inter-linked with or otherwise associated with a separate2.5D model rendering of the building and/or a 3D floor plan modelrendering of the building, etc., and including for a computer modeland/or floor map of a multi-story or otherwise multi-level building tohave multiple associated sub-floor models or maps for different storiesor levels that are interlinked (e.g., via connecting stairway passages).Accordingly, non-exclusive examples of an end user’s interactions with adisplayed or otherwise generated computer model (e.g., a 2.5D or 3Dmodel view that optionally includes images texture-mapped to walls ofthe displayed model) and/or 2D floor map of a building may include oneor more of the following: to change between a computer model view and afloor map view (collectively referred to herein as one or more mappingviews); to change between a mapping view and a view of a particularimage at a viewing location within or near the building’s interior; tochange the horizontal and/or vertical viewing direction from which acorresponding subset view of (or portal into) a panorama image isdisplayed, such as to determine a portion of a panorama image in a 3Dspherical coordinate system to which a current user viewing direction isdirected, and to render a corresponding planar image that illustratesthat portion of the panorama image without the curvature or otherdistortions present in the original panorama image; etc. Additionaldetails regarding embodiments of a system providing at least some suchfunctionality of a BIIP system (such as to provide or otherwise supportat least some functionality to an end user using a building informationviewer system and routine as discussed herein) are included in U.S.Non-Provisional Pat. Application No. 15/950,881, filed Apr. 11, 2018 andentitled “Presenting Image Transition Sequences Between ViewingLocations” (which includes disclosure of an ILTM system that is oneexample embodiment of a BIIP system generally directed to generating anddisplaying transitions between images captured at different viewinglocations); and in U.S. Provisional Pat. Application No. 62/911,959,filed Oct. 7, 2019 and entitled “Providing Simulated LightingInformation For Three-Dimensional Building Models” (which includesdisclosure of a BMLSM system that is one example embodiment of a BIIPsystem generally directed to generating and displaying simulatedlighting information in a three-dimensional model of a building’sinterior); each of which is incorporated herein by reference in itsentirety. For example, a computer model of an interior of a house orother building may be a 3D (three-dimensional) or 2.5D (two and a halfdimensional) representation that is generated after the house is builtand that shows physical components of the house’s actual interior (e.g.,walls, windows, doors, stairs, fireplaces, kitchen islands, cabinets,counters, lighting and/or plumbing fixtures and associated built-inelements such as sinks and showers/baths, curtains, wallpaper or paint,floor coverings, etc.), such as from analysis of images acquired in thehouse’s interior to reflect a current structure of the house (andoptionally non-fixed or temporary elements in the house, such asfurniture and/or furnishings). In some embodiments, automated operationsmay be performed that are involved in using images acquired at multipleassociated viewing locations (e.g., in multiple rooms or other locationswithin a building or other structure and optionally around some or allof the exterior of the building or other structure) to generate a 2Dfloor map for the building or other structure and/or to generate acomputer model for the building or other structure (e.g., a 3D modeland/or a 2.5D model), such as by analyzing visual information availablein the images, and for providing a representation of an interior of thebuilding or other structure (e.g., for subsequent use in generating andpresenting conditions for the interior of the building or otherstructure) - for example, in at least some such embodiments, suchtechniques may include analyzing one or more images taken in a room todetermine a shape of the room and/or to identify inter-room passages(e.g., doorways and other openings in walls) into and/or out of theroom. In at least some embodiments, an automated analysis of some or allimages may further be performed to determine corresponding informationabout the building interior, such as types of rooms, points of interestin particular locations, etc. In addition, a model of a buildinginterior may be displayed in at least some embodiments to a user in adisplayed GUI on a client computing device, and the user may be able tospecify via the GUI (or in another manner) at least some of theconditions for how the display is generated, such as one or more of thefollowing: one or more target times; an amount of the house or otherbuilding interior to display (e.g., one or more specific rooms, theentire interior, etc.); effects of changes to the building interior(e.g., adding or removing a window; adding or removing part or all of awall; changing furnishings or moveable elements; changing the colorand/or texture of a surface, such as a wall or a floor or a ceiling or acountertop; adding or removing lighting sources at specified locations;etc.); effects of changes outside the building interior (e.g., adding orremoving or changing a tree or other vegetation in an environmentsurrounding the building, such as in a yard of a house; adding orremoving or changing an exterior building or other external structure,whether on a same property as the building or a nearby property; etc.);a speed or rate at which an animation is displayed or other displayedinformation is changed; etc. In some embodiments, one or more types ofadditional information may be associated with and optionally displayedwith a computer 3D model (e.g., with full height informationrepresented) or computer 2.5D model (e.g., with partial representationsof height shown) of a building’s interior. As one example, one or moretypes of additional information about a building may be received,associated and displayed with such a model (e.g., with particularlocations in particular rooms) or otherwise accessible from thedisplayed model (e.g., upon selection by a user), such as one or more ofthe following: images; textual and/or audio annotations or otherdescriptions of particular rooms or other locations; other audioinformation, such as recordings of ambient noise; estimated room width,length and/or height dimensions; external information from anenvironment surrounding the building, such as about nearby buildingsand/or vegetation (e.g., as identified from satellite or other externalimages of the building, such as taken from the building and/or from anearby street or yard; as identified from data in one or more externaldatabases or other information sources, such as street maps or othergovernment records; etc.), whether by displaying the actual images or bygenerating and displaying visual representations of particular externalelements that are identified and modeled from images or other externalinformation sources; in-room images for a room that are projected on thewalls of the room shown in the model; geographical location and/ororientation information for the building; user-generated and/or‘crowd-sourced’ information provided by one or more end users about thebuilding, such as interior and/or exterior images, descriptions, etc.; a2D (two-dimensional) floor map of the building interior, such as usingan overhead schematic view (e.g., an orthographic top view); etc. Afloor map for a house may, for example, be presented to an end user in aGUI, such as with room labels added to some or all rooms (e.g., “livingroom”), room dimensions for some or all rooms, visual indications offixtures or appliances or other built-in features for some or all rooms(e.g., for the bathroom), visual indications of positions of additionaltypes of associated and linked information may be added (e.g., ofpanorama images and/or perspective images that an end user may selectfor further display, of audio annotations and/or sound recordings thatan end user may select for further presentation, etc., as shown in theliving room and master bedroom and kitchen/dining room), visualindications of doors and windows may be shown, etc. In addition, auser-selectable control 228 may be used to indicate a current floor thatis displayed for the floor map, and to allow the end user to select adifferent floor to be displayed - in some embodiments, a change infloors or other levels may also be made directly from the floor map,such as via selection of a connecting passage, such as the stairs tofloor 2 in the illustrated floor map. In addition, while not illustratedin FIG. 1 , in some embodiments the client computing devices 175 (orother devices, not shown) may receive and use generated computer modelsand/or other generated mapping-related information in additionalmanners, such as to control or assist automated navigation activities bythose devices (e.g., by autonomous vehicles or other devices), whetherinstead of or in addition to display of the generated information.

In the depicted computing environment of FIG. 1 , the network 170 may beone or more publicly accessible linked networks, possibly operated byvarious distinct parties, such as the Internet. In otherimplementations, the network 170 may have other forms, such as toinstead be a private network (such as a corporate or university network)that is wholly or partially inaccessible to non-privileged users. Instill other implementations, the network 170 may include both privateand public networks, with one or more of the private networks havingaccess to and/or from one or more of the public networks. Furthermore,the network 170 may include various types of wired and/or wirelessnetworks and connections in various situations.

FIG. 1 depicts a block diagram of an exemplary building interiorenvironment in which images are acquired and for which one or morecomputer models and/or 2D floor maps are generated, such as for furtheruse by the BIIP system to control the simultaneous and coordinatedpresentation of multiple types of information about the interior of thehouse, as discussed in greater detail with respect to FIGS. 2A-2X, aswell as for use in some embodiments in otherwise presenting the computermodels and/or floor maps and/or images to users. In particular, FIG. 1illustrates a first story of a multi-story building 198 (e.g., with apartial or full basement and/or a second story, not shown) with aninterior that was captured at least in part via multiple panoramaimages, such as by a mobile image acquisition device 185 with imageacquisition capabilities as it is moved through the building interior toa sequence of multiple viewing locations 210 (e.g., starting at viewinglocation 210A, moving to viewing location 210B along travel path 115,etc.). An embodiment of the ICA system (e.g., ICA system 160 on servercomputing system(s) 180, a copy 155 of some or all of the ICA systemexecuting on the mobile image acquisition device 185, etc.) mayautomatically perform or assist in the capturing of the datarepresenting the building interior, as well as to further analyze thecaptured data to generate panorama images to provide a visualrepresentation of the building interior, and an embodiment of the FMGMsystem (e.g., FMGM system 160 on server computing system(s) 180, a copy157 of some or all of the FMGM system executing on the mobile imageacquisition device 185, etc.) may automatically perform or assist in thegeneration of one or more computer models and/or a 2D floor maprepresenting the building interior. While such a mobile imageacquisition device may include various hardware components, such as acamera, one or more sensors (e.g., a gyroscope, an accelerometer, acompass, etc., such as part of one or more IMUs, or inertial measurementunits, of the mobile device; an altimeter; light detector; etc.), a GPSreceiver, one or more hardware processors, memory, a display, amicrophone, etc., the mobile device may not in at least some embodimentshave access to or use equipment to measure the depth of objects in thebuilding relative to a location of the mobile device, such thatrelationships between different panorama images and their viewinglocations may be determined in part or in whole based on features indifferent images, but without using any data from any such depthsensors, while in other embodiments one or more such depth sensors orother distance-sensing devices may provide depth data that is used incombination with other information from analysis of visual data ofcaptured images. In addition, while geographical orientation/directionalindicator 109 is provided in FIG. 1 for reference of the viewer, themobile device and/or ICA system and/or FMGM system may not use suchabsolute directional information in at least some embodiments, such asto instead determine relative directions and distances between viewinglocations 210 without regard to actual geographical positions ordirections in such embodiments.

In operation, the mobile image acquisition device 185 arrives at a firstviewing location 210A within a first room of the building interior (inthis example, in a living room accessible via an external door 190-1),and captures a view of a portion of the building interior that isvisible from that viewing location 210A (e.g., some or all of the firstroom, and optionally small portions of one or more other adjacent ornearby rooms, such as through doors, halls, stairs or other connectingpassages from the first room). The view capture may be performed invarious manners as discussed herein, and may capture information about anumber of objects or other features (e.g., structural details) that arevisible in images captured from the viewing location - in the example ofFIG. 1 , such objects or other features throughout the house include thedoorways 190 (including 190-1 and 190-3) and 197 (e.g., with swingingand/or sliding doors), windows 196 (including 196-1, 196-2, 196-3 and196-4), corners or edges 195 (including corner 195-1 in the northwestcorner of the building 198, corner 195-2 in the northeast corner of thefirst room, corner 195-3 in the southwest corner of the first room,corner 195-4 at the northern edge of the inter-room passage between thefirst room and a hallway, etc.), furniture 191-193 (e.g., a couch 191;chair 192; table 193; etc.), pictures or paintings or televisions orother hanging objects 194 (such as 194-1 and 194-2) hung on walls, lightfixtures, various built-in appliances or fixtures (not shown), othertypes of building information, etc. The user may also optionally providea textual or auditory identifier (or ‘label’ or ‘tag’) to be associatedwith a viewing location, such as “living room” for the room includingviewing locations 210A and/or 210B and/or identifiers “entry” and“living room -NE” for the viewing locations 210A and 210B, respectively,while in other embodiments the ICA system and/or FMGM may automaticallygenerate some or all such identifiers (e.g., by automatically analyzingvideo and/or other recorded information for a building to perform acorresponding automated determination, such as by using machinelearning) or the BIIP system may determine such identifiers (e.g., basedat least in part on input from BIIP system operator users and/or endusers) or the identifiers may not be used. For example, the BIIP, FMGMand/or ICA system may use machine learning or other techniques (e.g.,visual analysis techniques) to analyze video (e.g., in a near-real-timeor real-time manner with respect to its recording and/or presentation)to determine a textual tag/label or other identifier to be associatedwith each of one or more rooms or other areas shown in the video (oreach of one or more positions from which one or more image frames of thevideo are captured), such as by using one or more neural networks orother machine learning models that have been trained to associate suchtags/labels or other identifiers with visual data of video or otherimage sequences - such video may, for example, be part of a virtualshowing tour, whether for video being transmitted in a real-time ornear-real-time manner relative to its recording, or for a later playbackof such a prerecorded video. Additional details related to suchautomated determination of textual tags/labels or other identifiers forvisual data of one or more images (e.g., a video or other sequence ofimages) are included in U.S. Non-Provisional Pat. Application No.17/080,604 (now U.S. Pat. No. 11,164,361), filed Oct. 26, 2020 andentitled “Generating Floor Maps For Buildings From Automated Analysis OfVisual Data Of The Buildings’ Interiors”; U.S. Non-Provisional Pat.Application No. 17/013,323 (now U.S. Pat. No. 11,514,674), filed Sep.04, 2020 and entitled “Automated Analysis Of Image Contents To DetermineThe Acquisition Location Of The Image”; U.S. Non-Provisional Pat.Application No. 17/069,800, filed Oct. 13, 2020 and entitled “AutomatedTools For Generating Building Mapping Information”; U.S. Non-ProvisionalPat. Application No. 17/472,527, filed Sep. 10, 2021 and entitled“Automated Identification And Use Of Building Floor Plan Information”;U.S. Non-Provisional Pat. Application No. 17/150,958 (now U.S. Pat. No.11,252,329), filed Jan. 15, 2021 and entitled “Automated DeterminationOf Image Acquisition Locations In Building Interiors Using Multiple DataCapture Device”; U.S. Non-Provisional Pat. Application No. 17/201,996(now U.S. Pat. No. 11,481,925), filed Mar. 15, 2021 and entitled“Automated Determination Of Image Acquisition Locations In BuildingInteriors Using Determined Room Shapes”; U.S. Non-Provisional Pat.Application No. 17/185,793, filed Feb. 25, 2021 and entitled “AutomatedUsability Assessment Of Buildings Using Visual Data Of Captured In-RoomImages”; and U.S. Non-Provisional Pat. Application No. 17/459,820, filedAug. 27, 2021 and entitled “Automated Mapping Information GenerationFrom Analysis Of Building Photos”; each of which is hereby incorporatedby reference in its entirety.

After the first viewing location 210A has been adequately captured, themobile device 185 may move or be moved to a next viewing location (suchas viewing location 210B), optionally recording movement data such asvideo and/or other data from the hardware components (e.g., from one ormore IMUs, from the camera, etc.) during movement between the viewinglocations. At the next viewing location, the mobile device may similarlycapture a panorama image from that viewing location. This process mayrepeat for some or all rooms of the building and optionally external tothe building, as illustrated for viewing locations 210C-210J in thisexample. The acquired panorama images for each viewing location may befurther analyzed, including in some embodiments to render or otherwiseplace each panorama image in an equirectangular format, whether at thetime of image capture or later.

Various details are provided with respect to FIG. 1 , but it will beappreciated that the provided details are non-exclusive examplesincluded for illustrative purposes, and other embodiments may beperformed in other manners without some or all such details.

FIGS. 2A-2X illustrate examples of presenting a GUI that simultaneouslydisplays multiple types of information about an interior of a buildingin an integrated and coordinated manner, such as for the building 198and using images and their viewing/capture locations 210 discussed inFIG. 1 .

In particular, FIG. 2A illustrates an example embodiment of a GUI withvarious information 204 a, which is displayed in this example on amobile client device 205 (e.g., a smart phone or tablet), and whichshows multiple types of information about a common area of examplebuilding 198. The GUI in this example includes a primary pane 207 andtwo secondary panes 208 and 209, with each of the panes showinginformation of a different type about the same area of the example house198 shown in FIG. 1 . The primary pane 207 is showing an image takenfrom viewing location 210A in the living room in this example, with thedirection of the image being in the northeast direction (as shown byvisual indicator 109 a overlaid on the primary pane, as well as thevisual indicator 219 a shown in secondary pane 209). In this example,the primary pane 207 also includes a header 206 that provides adescription of the type of content shown (here, an image, as illustratedby the “photo gallery” label), as well as optionally having anadditional label specific to the photo or other image being displayed(here, “Living Room #1”).

The primary pane 207 may also have one or more user-selectable controlsthat enable the end user (not shown) to modify information beingdisplayed in the GUI, such as control 218 a in the header area 206 tochange the type of information being shown in the primary pane to adifferent type of content (e.g., by cycling through each of theavailable types of content). In addition, other possible user-selectablecontrols 201 are overlaid on the primary pane, such as a toggle control203 c to show or hide information about points of interest that arepresent in the current image (e.g., types of building informationdetermined in part or in whole from automated analysis of visual data ofcaptured images), a toggle control 203 d to show or hide a descriptionof the area of the house shown by the image, a toggle control 203 e toshow or hide information about questions and answers corresponding tothe area of the house shown in the image and to optionally allow the enduser to supply one or more additional questions or answers, a togglecontrol 203 b to switch to a so-called “AR” (or augmented reality) modein which tilting and rotating the body of the mobile device causes thedisplayed image to move accordingly (e.g., if the image shown in theprimary pane is a 360° panorama image with only a subset of it currentlyshown, such that tilting to the left causes an additional portion of thepanorama image to the left of the visible area to be shown, and with thecontrol 203 b not being shown if the content is not a panorama image), aselection control 203 a to switch to another image in the photo gallery(e.g., to cycle through all available images), etc. - as discussed ingreater detail elsewhere herein, in at least some embodiments andsituations, some or all of the displayed user-selectable controls may becontextual based on a type of content currently shown in the primarypane (in this example, associated with image content).

In addition to the image shown in the primary pane of the northeastcorner of the living room of the example house, the secondary panes 208and 209 show other types of content about the same location or area ofthe example house. In particular, the secondary pane 209 in this exampleshows a portion of a 3D computer model of the house, and in particularshows a portion of the computer model that includes the northeast cornerof the living room, as well as optionally including some or all of otherparts of the same floor - as previously noted, the computer model isoverlaid in this example with the visual indicator 219 a to illustratewhere the image shown in the primary pane was captured, and theorientation of the camera that captured the image. In addition, thesecondary pane 208 in this example shows a portion of an interface to aninteractive virtual tour of the house, with the interactive tour havinga plurality of inter-connected viewing/capture locations at which imagesand/or other information that are available for viewing or otherpresentation were captured. In this example, the interface includes animage that was also captured from viewing location 210A (e.g., apanorama image captured from that viewing location), with the interfacefurther including visual indicators 214 b and 214 c that each correspondto other viewing locations different from viewing location 210A (inparticular, to viewing locations 210B and 210C of FIG. 1 ,respectively) - as discussed in greater detail with respect to FIG. 2C,such visual indicators 214 may be user-selectable controls in at leastsome situations (e.g., when the interface to the interactive tour isshown in the primary pane) such that selection of one of the controlschanges the image displayed to change to one from the viewing locationassociated with the selected control.

It will be appreciated that a variety of types of user interactions mayoccur with example embodiments of the GUI, such as to enable an end user(not shown) to select a secondary pane and cause its content to be movedto the primary pane, with new contents of that secondary pane beingselected to correspond to the new primary pane contents (e.g., to movethe contents that were in the primary pane to that secondary pane, toselect a new content type to display in the secondary pane, etc.). Otheroptional user-selectable controls 202 are further shown in this examplefor reference purposes, although they may not be visible in someembodiments when three GUI panes are displayed as shown, such as if thecontrols 202 are associated with a different part of the GUI asdiscussed in greater detail with respect to FIG. 2F.

FIG. 2B continues the example of FIG. 2A, and in particular correspondsto the end user having selected the control 203 a in FIG. 2A to changethe image that is displayed in the primary pane to another image fromthe photo gallery. In this example, the new image displayed in theprimary pane 207 as part of the updated information 204 b in the updatedGUI is also taken from the same viewing location 210A, but in anorthwesterly direction (as illustrated by the updated visual indicator219 b shown in the 3D computer model of the secondary pane 209). In thisexample, the contents of the secondary pane 209 continue to be the sameportion of the 3D computer model as in FIG. 2A (with the updated visualindicator 219 b), which continues to correspond to the area of the houseshown in the primary pane of FIG. 2B, and the contents of the secondarypane 208 continue to be the same portion of the interactive tourinterface that correspond to the content of FIG. 2B as well (e.g., sincethe image in FIG. 2B continues to be taken from the same viewing/capturelocation as in FIG. 2A). It will be appreciated that the header portion206 of the primary pane 207 is also updated in FIG. 2B to reflect thenew image being shown in the primary pane, although in some embodimentssuch image-specific labels may not be used. It is also noted that theuser-selectable controls 202 mentioned in FIG. 2A are not shown in FIG.2B.

In addition, FIG. 2B further illustrates additional information that hasbeen overlaid on the primary pane 207, based on selection by the enduser of the user-selectable controls 203 c, 203 d and 203 e in thisexample, although it will be appreciated that the end user may insteadselect only zero, one, or two of those three controls in othersituations. In this example, selection of the user-selectable control203 c has caused several visual indicators 212 for points of interest inthe room to be illustrated, with visual indicator 212 b being currentlyselected and having a corresponding textual comment in the area 213 ofthe primary pane (to comment about paint on the walls of the room).Other visual POI indicators in this example include 212 a on thewest-facing picture window, 212 d on the north-facing window, and 212 con the overhead track lighting on the ceiling. While the selected visualindicator 212 b has associated text that is displayed in this example,other visual indicators for POIs may have other types of informationassociated, such as if the window visual indicators 212 a and/or 212 dhave images and/or video associated with them, such as to show images orvideos looking out the window (e.g., a time-lapse video over a 24-hourperiod of the exterior of the house from a window or door, such as toshow road traffic or people traffic outside over that time period). FIG.2P provides additional details about an example of automated analysis ofvisual data in an image captured in the building to identify types ofbuilding information to use for such POIs and/or to determineinformation to be displayed for such POIs, with additional details aboutsuch analysis included elsewhere herein. In addition to the activationof the control 203 c, the user-selectable control 203 d has also beenactivated to provide an audio description of the area shown in theimage, with the visual information 211 shown in this examplerepresenting audio information that may be audibly presented in responseto the selection of that control (e.g., instead of having a textualrepresentation of the information as shown in this example, or insteadin addition to the textual information, such as if a closed captioningoption is selected). Furthermore, the user-selectable control 203 bcorresponding to questions and answers is also selected, causingadditional information in the area 213 to be shown, such as a questionfrom another end user and a corresponding answer from a real estatelisting agent for the house, as well as further user-selectable controlsto allow the end user to ask a question to be answered, submit a commentfor display to others, or to cycle through other existing questions andcomments. It will be appreciated that illustrated types of informationmay be presented in other manners in other embodiments, or may not beshown.

FIG. 2C continues the examples of FIGS. 2A-2B, and illustrates anexample of the end user interacting with the GUI displayed in FIG. 2A toinitiate a switch of the content between the primary pane 207 and thesecondary pane 208, such that the primary pane 207 in the updatedinformation 204 c of the updated GUI of FIG. 2C is the interface to theinteractive tour of the house, with the visual indicators 214 b and 214c in FIG. 2C now being selectable by the user to change the currentviewing location, as further illustrated with respect to FIG. 2D. In atleast some embodiments and situations, the information displayed in thesecondary pane 209 would not change in FIG. 2C (other than updating thevisual indicator 219 c), such as due to the content of the primary panecontinuing to correspond to the same portion of the 3D computer modelthat was shown in FIGS. 2A and 2B - in this example, however, theinformation in the secondary pane 209 has been updated for the purposesof illustration to show the entire 3D computer model for the currentfloor of the house. In addition, contents of the secondary pane 208 havebeen updated in this example to show the image that was previouslypresent in the primary pane in FIG. 2A. It will be appreciated that theheader portion 206 of the primary pane 207 is also updated in FIG. 2C toreflect the new type of content being shown in the primary pane.

FIG. 2D continues the examples of FIGS. 2A-2C, and in this examplecorresponds to the end user having selected the user-selectable visualindicator 214 b in FIG. 2C to change the current viewing location forthe interactive tour. In response to that selection, the backgroundimage used for the interactive tour interface in the primary pane 207 ofFIG. 2D has changed to the updated information 204 d shown in theprimary pane for the updated GUI, corresponding to an image taken fromthe viewing location 210B in the northeast part of the living room, andthe user-selectable visual indicators in the primary pane have similarlybeen updated, with the direction of the previous indicator 214 cchanging to point to the direction of the hallway from the newbackground image that is shown, and with the previous visual indicator214 b for the northeast living room viewing location 210B (which is nowthe current viewing location) being changed to the new user-selectablevisual indicator 214 a for the entry viewing location 210A in the livingroom (which was the previous viewing location). In addition, a newvisual indicator 203 f has been added to reflect an alternative mannerfor the end user to adjust the panorama image shown as the background ofthe primary pane, such as to manually drag the image up, down, leftand/or right to display other parts of the panorama image that are notcurrently visible in the subset shown in the primary pane, as discussedfurther in FIG. 2E. As in the previous examples, the visual indicator219 d in the 3D computer model of the secondary pane 209 has beenupdated to correspond to the location and orientation at which the newbackground image shown in the primary pane was captured. In addition,while the content previously shown in the secondary pane 208 of FIG. 2Cmay in some embodiments and situations continue to be shown in thesecondary pane 208 in FIG. 2D, in this example the content has beenchanged for illustrative purposes to provide content of another typethat corresponds to the same area of the house, which in this example isa video of at least that portion of the house. In this example, thevideo in the secondary pane 208 includes a visual indicator 203 g of aplay button for the video, although the visual indicator may not beuser-selectable in at least some embodiments until the video content ismoved to the primary pane.

FIG. 2E continues the examples of FIGS. 2A-2D, and in this exampleillustrates the effects of the end user having used the control 203 f ofFIG. 2D to change the subset of the background panorama image (e.g., a360° panorama image) that is shown by rotating approximately 180°, so asto now point toward the southwest corner of the living room from theviewing location 210B in the northeast corner of the living room, asshown in the updated information 204 e of the updated GUI. In thisexample, the user-selectable visual indicators 214 c and 214 a (forviewing locations 210C and 210A, respectively) have been updated tocorrespond to the positions of those viewing locations relative to thecontent shown in the background image, and the visual indicator 219 e inthe 3D computer model of the secondary pane 209 has been similarlyupdated. While the contents of the secondary pane 208 may be unchangedin this situation, the image in the photo gallery of secondary pane 208has been changed in this example to an image that is closest to that ofthe current background image in the primary pane (so as to maintain acoordinated display of information that is integrated with the primarypane), which in this example is of the window of the west wall of theliving room. In addition, this example further illustrates thatadditional types of information may be overlaid on images in at leastsome situations, such as in this example to overlay simulated lightinginformation 225 a on that image (although a correspondinguser-selectable control to initiate that display is not shown in thisexample). It will be appreciated that additional overlay information,such as that of the simulated lighting, may be controlled by the enduser in various manners, including as is discussed in greater detailwith respect to FIG. 2F.

FIG. 2F continues the examples of FIGS. 2A-2E, and illustrates updatedinformation 204 f in the updated GUI in which the secondary panes 208and 209 are not shown, but additional user-selectable controls 202 areincluded (e.g., based on the primary pane 207 being changed to show the3D computer model of the house). In this example, the header information206 is updated to include an additional user-selectable control 218 bthat allows the end user to select which floor of the house is shown,with the house in this example having an additional basement flooraccessible via the stairs (as illustrated further with respect to FIG.2G). In this example, the 3D computer model includes illustrations ofthe positions of the viewing/capture locations for the interactive tour,with the visual indicator 225 b being added to correspond to the currentviewing location that was last selected. This example further includes auser-selectable control 225 c that allows an end user to orient thecomputer model of the house in different manners according to differentspecified criteria, such as to have a top of the primary pane correspondto magnetic or geographic north, have the top or bottom of the primarypane correspond to the main entry of the house, to provide a best fit ofthe 3D computer model to the current size and shape of the primary pane,etc. In addition, the user-selectable controls 202 may enable varioustypes of additional information to be overlaid on the floor map or tootherwise invoke additional functionality to be provided, such as one ormore of the following: control 202 a activates a default mode thatallows the end user to move to a detailed view of a particular room orviewing location by selecting that room or viewing location on the 3Dcomputer model; control 202 b shows additional descriptive informationof one or more types on the computer model, such as one or more types ofPOIs to show one or more types of building information (e.g., asselected by the user, based on available types of building information,etc.); control 202 c shows measurements or other scale information onthe 3D computer model; control 202 d shows simulated sunlight on the 3Dcomputer model for specified conditions (e.g., one or more times of dayand/or times of year); control 202 e shows actual interior lighting,optionally under specified conditions; control 202 f shows informationabout a surrounding environment of the building (with FIG. 2G providingone example); control 202 g provides sound recordings from one or morelocations of the building (e.g., of ambient sound at specified times);control 202 z allows virtual objects to be added to the 3D computermodel and/or surfaces in the 3D computer model to have virtual changes(e.g., to change color, texture etc. of walls, floors, object surfaces,etc.); etc. - it will appreciated that various other types of controls202 may be provided in other embodiments, and/or that some of theillustrated types of controls 202 may not be used.

FIG. 2G continues the examples of FIGS. 2A-2F, and illustrates anexample of the GUI being updated with information 204 g to again includeonly a single primary pane 207, which in this example has no header orseparate area for user-selectable controls - in addition, theinformation being displayed has been adjusted to reflect that the device205 has been rotated to a landscape orientation. In this example, theend user has selected to display information about the surroundings ofthe building, which in this example includes a street to the west of thehouse, with two additional houses across the street being shown, as wellas additional information on the house’s property that includes twotrees, sidewalks, a driveway and garage, and bushes or other shrubs onthe south side of the house (as indicated by the visual indicator 109g). This example further illustrates a deck to the east of the house,and includes visual information about a basement existing below groundlevel, although details of the interior of the basement are not shown inthis example. The vegetation and surrounding buildings are shown in thisexample using simplified geometrical shapes, such as to enable the useof such shapes to simulate shading (not shown) if a simulated lightinginformation option is selected for the display, although actual imagesof surroundings may instead be used in other embodiments and situations.It will be appreciated that a variety of other types of information maybe shown about the surroundings of the building, including images and/orvideo from ground level and/or an aerial view, as well as additionalpoints of interest in descriptive information about the surroundingneighborhood.

FIG. 2H continues the examples of FIGS. 2A-2G, and illustrates anexample of the GUI similar to that of FIG. 2F, but updated withinformation 204 h corresponding to selection of control 202 h by the enduser. In this example, the control 202 h corresponds to updating thedisplayed floor plan of the house to visually show information aboutspaces of the house that are public (e.g., living room, kitchen, diningroom, hallways, stairs, etc.) and that are private (e.g., bathrooms,bedrooms, etc.), such as displaying one or both types of spaces withspecified colors and/or patterns. In this example, a legend 221 h isdisplayed to show options 222 h for patterns to use, with the displayedfloor plan having corresponding displayed information (e.g., via apublic/private visual layer that is overlaid on the floor plan). Thepublic and private spaces of the house may be determined in variousmanners in various embodiments, such as automatically (e.g., based onpreviously determined room types, based on analysis of images) and/orbased on user annotations or other input (e.g., from one or more systemoperator users of the BIIP and/or ICA systems, from end users, etc.).

FIG. 2I continues the examples of FIGS. 2A-2H, and illustrates anexample of the GUI similar to that of FIG. 2H, but instead being updatedwith information 204 i (instead of information 204 h) corresponding toselection of control 202 i by the end user. In this example, the control202 i corresponds to updating the displayed floor plan of the house tovisually show icon information about types of rooms of the house and/orassociated types of room functionality (e.g., instead of the room typetextual labels shown in FIG. 2H). In this example, a legend 221 i isdisplayed to show icons 222 i to use for different types of rooms, withthe displayed floor plan having corresponding displayed icons (e.g., viaa room type icon visual layer that is overlaid on the floor plan, suchas instead of or in addition to a room type textual label visual layer).As previously discussed, types of rooms may be determined in variousmanners in various embodiments, such as automatically (e.g., based onanalysis of images and/or use of machine learning techniques) and/orbased on user annotations or other input (e.g., from one or more systemoperator users of the BIIP and/or ICA systems, from end users, etc.).

FIG. 2J continues the examples of FIGS. 2A-2I, and illustrates anexample of the GUI similar to that of FIG. 2I, but being updated withinformation 204 j corresponding to selection of control 202 j by the enduser. In this example, the control 202 j corresponds to updating thedisplayed floor plan of the house to visually show examples ofinhabitant flow pattern information 221 j corresponding to typical orcommon movement patterns of people (e.g., in addition to room typeicons, while in other embodiments may be shown without room type labelor icon information). In this example, the flow pattern information maybe displayed, for example, via a flow pattern visual layer that isoverlaid on the floor plan, such as instead of or in addition to one ormore other visual layers. Such flow pattern information may bedetermined in various manners in various embodiments, such asautomatically (e.g., based on analysis of video taken over time withinthe house interior, based on other tracking of user movements within thehouse interior, based on analysis of images to determine open lanes orspaces for user movement, etc.) and/or based on user annotations orother input (e.g., from one or more system operator users of the BIIPand/or ICA systems, from end users, etc.).

FIG. 2K continues the examples of FIGS. 2A-2J, and illustrates anexample of the GUI similar to that of FIG. 2H, but updated withinformation 204 k corresponding to selection of control 202 k by the enduser. In this example, the control 202 k corresponds to updating thedisplayed floor plan of the house to visually show information aboutspaces of the house that are likely to be of most interest to the enduser, such as displaying spaces corresponding to different levels oflikely user interest with different specified colors and/or patterns. Inthis example, a legend 221 k is displayed to show patterns 222 k forpatterns to use, with the displayed floor plan having correspondingdisplayed information (e.g., via a user-specific personalized visuallayer that is overlaid on the floor plan). The areas of the house thatare likely to be of interest to the end user may be determined invarious manners in various embodiments, such as automatically (e.g.,based on tracking a quantity of time and/or a length of time that theend user has previously spent viewing different parts of this house,based on an analysis of corresponding types of information for the enduser from multiple other houses, etc.) and/or based on user annotationsor other input (e.g., from previously specified preferences of the enduser; from previous indications of the end user of particular parts ofthe house that are of interest, such as by flagging or pinningparticular rooms and/or images; etc.).

FIG. 2L continues the examples of FIGS. 2A-2K, and illustrates anexample of the GUI similar to that of FIG. 2F, but updated withinformation 204I corresponding to selection of control 202I by the enduser. In this example, the control 202I corresponds to providinginteractive functionality for the end user to specify a length on thedisplayed floor plan of the house (e.g., by dragging a line 223I, byselecting two end points, etc.) and receive information about thedistance of that length (e.g., displayed distance information 224I). Thedistance may be automatically determined in various manners in variousembodiments, such as based on previous and/or concurrent analysis ofimages of the house interior to determine sizes of known or unknownobjects, as discussed in greater detail elsewhere herein. In addition,while the distance functionality is provided using a 3D floor plan inthis example, similar functionality may be provided on other media ortypes of information in other embodiments, such as on displayed panoramaor perspective images, during interactive virtual tours of inter-linkedviewing locations, etc.

FIG. 2M continues the examples of FIGS. 2A-2L, and illustrates anexample of the GUI updated with information corresponding to selectionof control 202 m by the end user. In this example, the control 202 mincludes a time-based slider (and associated information) 226 m thatcontrols what types of information are displayed in the one or morepanes, which in this example includes user-selectable indicators 223 mcorresponding to viewing/capture locations at which images and/or otherdata is captured in the building, and such as based on the acquisitiontimes for those images and/or other data. In comparison to theviewing/capture locations illustrated in FIG. 2L, the viewing/capturelocations include two acquisition locations (in this example, tocorresponding to viewing/capture locations outside of the house, whichare captured at a later time after the images and/or other data capturedat the viewing/capture locations inside the house), and theuser-selected time corresponds to a current time (such as to shown allacquisition locations for the building).

FIG. 2N continues the examples of FIGS. 2A-2M, and illustrates anexample of the GUI that is further updated with informationcorresponding to modification of the time-based slider control andassociated information change 226 n by the user, as shown in updatedinformation 202 n. In this example, the time has been changed to anearlier date, and one of the viewing/capture locations 223 m that wasshown in FIG. 2M is no longer displayed (as shown with indicator 223 n)due to being acquired after the currently selected time.

FIG. 2O (also referred to as “2-O” herein to differentiate from thenumber 20) continues the examples of FIGS. 2A-2N, and illustrates anexample of the GUI in which the 3D computer model shown in FIG. 2N hasbeen updated to a 2D floor map model of the same floor (the currentlyselected main floor, or “Floor 1”), and is further updated withinformation corresponding to further modification of the time-basedslider control and associated information 226 o by the user, as shown inupdated information 202 o. In this example, the time has been changed toan earlier date before such remodeling of the house had been completedand before some of the previously displayed viewing/capture locationswere used to acquire images or other data. In this example, visualindicators 223 o indicate the absence of viewing/capture locations thatwere shown at the later time corresponding to FIG. 2N, and additionalstructural areas 227 o and 228 o are shown that were removed by thelater time corresponding to FIG. 2N. It will be appreciated that othertimes of information may similarly be determined for a house or otherbuilding at different times (e.g., with different corresponding computermodels and associated additional information), with the display of suchbuilding information changing based on a currently selected time. Asdiscussed elsewhere herein, various types of changes over time may beillustrated in other manners, including with animation or other changesplayed over a period of time, including with respect to the types ofinformation discussed in FIG. 2O and elsewhere herein.

FIG. 2P continues the examples of FIGS. 2A-2O, and illustrates anexample of an image 250 p that is acquired in the living room of thehouse, with various types of building information 199 that may beautomatically determined via analysis of visual data of the image, suchas for use in POIs displayed in the GUI (e.g., POIs similar to thosediscussed with respect to FIG. 2B). In this example, the types ofbuilding information may include an outlet 199 a for one or more typesof wiring (in this example, a combined electrical and cable outlet),wall surface 199 b (e.g., type of material, such as wallpaper or paint;color; texture; etc.), window type and/or associated furnishings 199 cfor window 196-2 (e.g., a picture window, a window that opens byswinging inwards, window curtains or drapes, etc.), other furnishings199 d (e.g., a tapestry on a wall), furniture 199 e (e.g., built-infurniture, such as a window seat; moveable furniture; etc.), lightingfixture type 199 f, piping or other conduit information 199 g (e.g., tocorrespond to an HVAC air vent), etc. It will be appreciated thatvarious other types of building information may be determined in othersituations, including for other types of rooms such as bathrooms (e.g.,sinks/bathtubs/showers and associated types of plumbing fixtures, floortype, counter type, types of lighting fixtures, etc.), kitchens (e.g.,sinks and associated types of plumbing fixtures, floor type, countertype, cabinet types, types of lighting fixtures, kitchen islands and/orother build-ins, appliances, etc.), utility rooms (e.g., sinks andassociated types of plumbing fixtures, floor type, counter type, typesof lighting fixtures, appliances, etc.), etc., as discussed in greaterdetail elsewhere herein.

FIG. 2Q continues the examples of FIGS. 2A-2P, and illustrates anexample of the GUI updated with information corresponding to selectionof control 202 q by the end user. In this example, the control 202 q isa slider that controls what types of information about infrastructureare overlaid on information displayed in the one or more panes, such aswith respect to wiring on one or more types, piping or other conduits,etc., and such as may be further specified using a user-selectable‘settings’ control 226 q and/or based on previously selected orotherwise specified information (e.g., user preferences) - in otherembodiments, one or more toggle controls may instead be used to turn onor off the display of information about one or more types ofinfrastructure (or other types of building information displayed asPOIs). In this example, the use of the control causes types of buildinginformation discussed with respect to FIG. 2P to be displayed, includinga visual indicator 233 q corresponding to an air vent and a visualindicator 232 q corresponding to an electrical outlet. It will beappreciated that the selected type(s) of infrastructure may be displayedthroughout the portions of the house currently shown in the GUI based onanalysis of visual data of corresponding images, including to beoverlaid on other types of underlying information (e.g., on images,interactive tours, 3D computer models, 2D floor map models, etc.), andin some cases may be displayed with a user-selectable visual indicatorto enable a particular POI to be selected to cause additionalinformation to be displayed (e.g., one or more images from which theinformation for the POI was determined, additional types of informationabout the type of infrastructure corresponding to a selected POI, etc.).

FIG. 2R continues the examples of FIGS. 2A-2Q, and illustrates anexample of the GUI updated with information corresponding to selectionof control 202 r by the end user. In this example, the control 202 r isa slider that controls virtual removal of information that wouldotherwise be present in the information for the building displayed inthe one or more panes, such as with respect to one or more of walls orother structural elements, furnishings, furniture, fixtures, etc., andsuch as may be further specified using a user-selectable ‘settings’control 226 r and/or based on previously selected or otherwise specifiedinformation (e.g., user preferences) - in other embodiments, one or moretoggle controls may instead be used to turn on or off the virtualremoval of one or more types of information. In this example, the use ofthe control causes building information to be virtually removed thatincludes the pantry 229 r (e.g., as part of a consideration to remodelthe kitchen and to use that space for another purpose), bathroomfixtures 227 r, etc. It will be appreciated that the virtual removal ofbuilding information may be performed throughout the portions of thehouse currently shown in the GUI, including for other types ofunderlying information (e.g., on images, interactive tours, 3D computermodels, 2D floor map models, etc.), and that the selected type(s) ofvirtually removed information may in some cases be indicated with auser-selectable visual indicator to enable selection to cause additionalinformation to be displayed (e.g., about the virtually removedinformation, etc.).

FIG. 2S continues the examples of FIGS. 2A-2R, and illustrates anexample of the GUI updated with information corresponding to selectionof control 202 s by the end user. In this example, the control 202 s isa slider that controls virtual addition of information that wouldotherwise not be present in the information for the building displayedin the one or more panes, such as with respect to one or more of wallsor other structural elements, furnishings, furniture, fixtures, etc.,and such as may be further specified using a user-selectable ‘settings’control 226 s and/or based on previously selected or otherwise specifiedinformation (e.g., user preferences) - in other embodiments, one or moretoggle controls may instead be used to turn on or off the virtualaddition of one or more types of information. In this example, the useof the control causes building information to be virtually added thatincludes various kitchen information 231 s (e.g., as part of aconsideration to remodel the kitchen), such as to add a kitchen islandwith a cooktop, to add a table and chairs, to add a new type of floorcovering, etc., and further includes a structural addition 227 s (e.g.,to add a master bathroom to the master bedroom by taking space from theliving room). It will be appreciated that the virtual addition ofbuilding information may be performed throughout the portions of thehouse currently shown in the GUI, including for other types ofunderlying information (e.g., on images, interactive tours, 3D computermodels, 2D floor map models, etc.), and that the selected type(s) ofvirtually added information may in some cases be indicated with auser-selectable visual indicator to enable selection to cause additionalinformation to be displayed (e.g., about the virtually addedinformation, etc.).

FIG. 2T continues the examples of FIGS. 2A-2S, and illustrates anexample of the GUI updated with information corresponding to selectionof control 202 t by the end user, such as in a manner somewhat analogousto that previously illustrated in FIGS. 2H and 2I. In particular, inthis example, a type of room and/or associated functionality 221 t hasbeen selected (e.g., from a list 222 t of options), which in thisexample corresponds to a bedroom. A corresponding partial floor plan ofthe house is shown with those room types and/or functionality typeshighlighted, including in this example to show partial floor plans foreach of the floors having a corresponding room/functionality type.

FIG. 2U continues the examples of FIGS. 2A-2T, and illustrates anexample of the GUI in which a visual listing of multiple types ofinformation about a selected house is shown, corresponding to selectionof control 202 u by the end user. In this example, the displayedinformation includes an array of images captured for the house (e.g.,photo 1, photo 2, etc.) and other types of building information, that inthis example include a street view of the house, a 3D computer model ofthe house, a 2D floor map model of the house, a video captured at thehouse, etc. - it will be appreciated that other types of buildinginformation may similarly be shown, such as an interactive tour, othervideos, animations, non-visual data (e.g., audio recordings), etc. Inthis example, each image or other type of building information shown inthe array may be selectable by the user, such as for that selectedbuilding information type to become the primary (or only) type ofinformation shown in an updated GUI display. It will be appreciated thatvarious other types of information may similarly be available via suchan interface, such as textual information (e.g., facts about thebuilding; a map of a surrounding area; an asking price for a buildingthat is available for sell or other type of possible acquisition, suchas rental; an estimated value for a building for a sale or other type ofpossible acquisition, such as based on an automated determination usinga comparison to data about other similar buildings or in another manner;etc.).

FIG. 2V continues the examples of FIGS. 2A-2U, and illustratesinformation similar to that of FIG. 2D, but in which video informationsuch as that shown in the secondary pane 208 of FIG. 2D has been movedto the main pane 207 of FIG. 2V, and in which the secondary pane 208 ofFIG. 2V shows information about an interactive tour in a manner similarto that of FIGS. 2A and 2B - various information 204 v is displayed inthe GUI of FIG. 2V, including a header area 206 that illustrates thetype of content shown in the primary pane 207. In particular, in theexample of FIG. 2V, the video information shown in the main pane 207corresponds to a live virtual showing tour of the house 198 that occursin a real-time or near-real-time manner with respect to the recordingand/or transmitting of the video, with a presenter device (not shown)currently positioned in the living room and facing in a roughlynorthwest direction and rotating toward the north as well as laterallymoving in a roughly eastward direction through a succession of multiplelocations inside and/or around an outside of the building - in thisexample, motion artifacts are shown in FIG. 2V to denote changes in thedisplayed visual data corresponding to movement of the camera on thepresenter device, but it will be appreciated that the video displayed toan actual recipient on a client device may not include such artifacts.In the illustrated example, audio data may be included as part of thevideo, with example information 211 corresponding to spoken information(e.g., by a user of the presenter device, such as captured by amicrophone of the presenter device, or by another participant in aninteractive communication session amongst multiple client devices thatare receiving the video transmission) that is transmitted as part of thevideo to the various recipient participants and presented on the clientdevices of the participants, while in other embodiments and situationssuch audio may not be included as part of the video.

In this example, the FMGM system is performing automated operations thatinclude analyzing visual data of the video to determine one or moretypes of information, such as a tag/label or other identifier 215 acorresponding to the room or other area in which the video is currentlybeing recorded, and/or one or more tags/labels or other identifiers 216(e.g., 216 a and 216 b) corresponding to features identified in thecurrent room or other area - as discussed in greater detail elsewhereherein, such automated determination of tags/labels or other identifiersmay in some embodiments (such as the illustrated example) be performedin a real-time or near-real-time manner with respect to the captureand/or transmission of the visual data, such as within milliseconds,seconds, minutes, etc. In addition, the primary pane 207 also includesvarious user-selectable GUI controls, including controls 203 thatcorrespond to the video playback and recording (e.g., a play control 203g, a fast-forward control 203 h, etc.), with most of the playbackcontrols being currently grayed out and not selectable due to the videobeing live rather than a playback of an earlier recorded video, althougha recording control is currently selectable to enable one or moreparticipants in the interactive communication session to record thevideo (e.g., just visual and audio data transmitted from the presenterdevice, to include some or all of the additional data communicatedbetween participants as part of the interactive communication session,etc.). As discussed with respect to the GUI information elsewhereherein, the information shown in the illustrated panes of the GUI may becoordinated, such as to update the information overlaid on the computermodel of the building in pane 209 to reflect a current position of thepresenter device that is recording video shown in the primary pane 207,to update the portion of the building shown in the computer model and/orinteractive tour of secondary panes 208 and 209 to correspond to thatsimilar position of the presenter device, to display and/or highlightparticular overlaid information (e.g., tags/labels and otheridentifiers) corresponding to a current room to which the videocurrently corresponds, etc.

As one non-exclusive embodiment, the video displayed is part of avirtual showing tour of a building by a presenter device at thebuilding, such as to provide a shared virtual showing tour to otherclient devices remote from the building. Such a virtual showing tourmay, for example, be conducted using video conference functionality ofthe FMGM system (or separate video conference software used inconjunction with the FMGM system). The presenter device (such as a smartphone, a tablet computer, a laptop computer, etc.) has a camera that isaimed at various features of the property, while the other participantsof the virtual showing tour may concurrently view the features of theproperty on their own electronic client communication devices at theirown locations respectively. The transmissions from the presenter device,and between the other client devices as part of an interactivecommunication session, may include audio, video and/or other types ofdata (e.g., feedback and instructions, and referred to generally hereinas ‘digital data’) - in at least some embodiments and situations, someor all of the transmissions may be relayed via one or more intermediateserver devices that implement at least some of the FMGM system, referredto at times herein as ‘relay servers’.

The illustrated GUI controls in FIG. 2V also include various GUIcontrols 1010-1062, such as to provide functionality related to theinteractive communication session for the virtual showing tour. In thisexample, a first user indicator is shown at 1010 (e.g., corresponding toa first user using a client device that is a recipient of the virtualshowing tour video, such as a seller of the building and its property),and video images of the first user are captured by the first user devicein this example and transmitted to the current device (either directlyor through the media relay server), and displayed at 1010. In addition,the first user indicator 1010 in this example includes a first useraudio control 1012 and a first user video control 1014, such asuser-selectable buttons that a user of the current device can operate toturn on or off the audio and video of the first user respectively (e.g.,audio of the first user as captured by a microphone of the first user’device and sent to the current device either directly or through themedia relay server when the audio control 1012 is enabled) -audio of thefirst user that is enabled and received is played back via a speaker ofthe current device, but when the audio control 1012 is disabled (meaningturned off), the audio of the first user is not sent to the currentdevice or not played back on the current device. A second indicator isshown at 1040 (e.g., corresponding to a second user using a clientdevice that is a recipient of the virtual showing tour video, such as apotential buyer of the building and its property), but the second useris not shown in the indicator 1040 in this example, with an anonymousicon shown (e.g., to hide an identity of the second user, includinginformation such as the second user’s face, body, setting, etc.) - forexample, video images of the second user captured by the second user’sdevice are not sent to the current device or are not displayed on thecurrent device. In this example, the current user of the current deviceand the second user cannot speak to or hear from each other either, withthe muted audio status icon 1042 indicating that fact and not being anoperable control. A fourth indicator of the current fourth userherself/himself is shown at 1020, with video images of the currentfourth user herself/himself being captured by the camera of the currentdevice and shown at 1020, such as to show what other participants of thevirtual showing tour video and its interactive communication session,and audio of the fourth user is captured using a microphone of thecurrent device. A third indicator is shown at 1030 (e.g., correspondingto a third user using a client device that is a recipient of the virtualshowing tour video, such as a buyer’s third user of the building and itsproperty), with video images and audio of the third user being capturedby a camera and microphone of the third user’s device respectively, andtransmitted to the current device either directly or through the mediarelay server, with the video images of the third user displayed at thethird user indicator 1030 and audio of the third user being played backthrough a speaker of the current device. The third user indicator 1030also includes a third user audio control 1032 and a third user videocontrol 1034, which allow the current fourth user to enable or disablethe corresponding media data (audio or video) from the third user’sdevice. When the controls 1032-1034 are disabled, the correspondingmedia data is not transmitted to the current device or not presented onthe current device, and when the controls 1032-1034 are enabled, thecorresponding media data is transmitted to the current device andpresented on the current device.

In this example, the FMGM system (e.g., a special remote computersoftware application of the FMGM system) further provides a terminationcontrol 1050, a real-time audio feedback control 1052, an engagementcontrol 1054, an audio mute control 1058, a video mute control 1060 anda confidential audio control 1062. When the termination control 1050 isselected, the FMGM system terminates the current fourth user’sparticipation in the virtual showing tour, so as to exit from thevirtual showing tour. When the real-time audio feedback control 1052 ispressed and held, the current fourth user speaks out her/his request tothe third user or another indicated user (e.g., another participant inthe interactive communication session with which a confidential audiopair is established, such as based on a preexisting relationship betweenthe current user and that other user and/or based on respective roles ofthe current user and that other user) for her/his feedback on aparticular room or other feature (such as the dining room, the kitchen,etc.) being shown and indicated in the video (also referred to herein asthe active feature). The FMGM system receives the audio data,representing the current fourth user’s real-time feedback request andcaptured by the microphone of the current device - the audio is nottransmitted to or presented on devices other than that of the third useror other indicated user, with the FMGM system optionally transcribingthe audio into text and displaying it on the display screen of thecurrent device, as well as sending that text and/or the audio to thedevice of the third user (or of the other indicated user) eitherdirectly or through a relay server. When the device of the third user(or of the other indicated user) receives the real-time feedbackrequest, it is presented to the third user (or the other indicateduser), such as to display text on that device’s display screen or toplay back audio via that device’s speaker. In response, the third user(or the other indicated user) may orally provide a response, with amicrophone of the device of the third user (or the other indicated user)capturing the audio of the real-time feedback response device - theaudio is not transmitted to or presented on devices other than that ofthe current device that sent the real-time feedback request, with theFMGM system optionally transcribing the audio into text and displayingit on the display screen of the device of the third user (or the otherindicated user), as well as sending that text and/or the audio to thecurrent device either directly or through a relay server. The FMGMsystem may further store the real-time feedback request and response(referred to herein as virtual showing tour real-time feedback) and useit in one or more further manners, such as to update listing informationassociated with the building and its property. The real-time audiofeedback mode may be ended in various manners, such as when thereal-time audio feedback control 1052 is released, or alternatively whenthe real-time audio feedback control 1052 is pressed and released asecond time (in which case the real-time audio feedback control 1052 isused as a toggle button). When the engagement control 1054 is selected,the FMGM system provides engagement data of the first user (or otherindicated user), and displays the same on the display screen of thecurrent device. When the recording control 203 i is selected, the FMGMsystem starts recording of the virtual showing tour, and stops therecording when indicated (e.g., via toggling of the recording control,when the stop control is selected, etc.), with the recording of thevirtual showing tour and its associated media being saved for later use.When the audio mute control 1058 is turned on, the FMGM system does nottransmit any audio of the current fourth user to the other devices, anddoes transmit such audio as captured by the microphone of the currentdevice to the other devices when turned off, although the audio of thecurrent fourth user may not be transmitted or relayed to a device thathas disabled the audio of the current fourth user via corresponding GUIcontrols on that device. Alternatively, pressing the audio mute control1058 may bring up a selection list that allows the current fourth userto select which other users for which to mute or unmute the audio feed.When the video mute control 1060 is turned on, the FMGM system does nottransmit any video images of the current fourth user to the otherdevices, and does transmit such video as captured by the camera of thecurrent device to the other devices when turned off, although the videoof the current fourth user may not be transmitted or relayed to a devicethat has disabled the video of the current fourth user via correspondingGUI controls on that device. Alternatively, pressing the video mutecontrol 1060 may bring up a selection list that allows the currentfourth user to select which other users for which to mute or unmute thevideo feed. When the confidential audio control 1062 is turned on, aconfidential audio communication channel between the current fourth userand the first user (or other indicated user) is established. In theconfidential audio mode, the audio of the current fourth user is onlysent to the device of the first user (or other indicated user) eitherdirectly or through a media relay server - when the confidential audiocontrol 1062 is turned off, the confidential audio mode is then turnedoff; and the audio of the current fourth user is no longer limited tothe device of the first user (or other indicated user), and istransmitted to all of the other devices (other than to a device that hasdisabled the audio of the current fourth user via corresponding GUIcontrols on that device).

FIG. 2W continues the examples of FIGS. 2A-2V, and illustratesinformation similar to that of FIGS. 2F and 2H through 2-O, but in whichvideo information (e.g., as part of a virtual showing tour, such as in amanner similar to that discussed for FIG. 2V and elsewhere herein) isillustrated in secondary pane 208, with a computer model of the firststory of the building being shown in primary pane 207. An illustratedpath 221 w is overlayed on the computer model in the primary pane toreflect movement of a presenter device through a succession of multiplelocations at the building while capturing the video for the virtualshowing tour - as discussed with respect to FIG. 2V and elsewhereherein, the video may be live (e.g., transmitted to the current clientdevice and optionally other devices in real-time or near-real-time asthe presenter device moves through the building) or being played backfrom a previous recording, and in this example is a playback of recordedvideo that allows a prior analysis of the visual data of the video (andoptionally additional data captured by the presenter device, such asmovement data from IMU sensors) to determine the path 221 w and tofurther show a portion of a future part of the path while the presenterdevice is still in the living room. In this example, various information204 w is displayed in the GUI, including various user-selectable GUIcontrols 202 that include the currently selected control 202 wcorresponding to displaying video for a virtual showing tour, andtags/labels or other identifiers 215-217 determined for rooms or otherareas of the building, structural elements and other features of suchrooms and other areas, attributes of rooms or other areas or thebuilding as a whole, etc. and overlaid on the computer model - in thisexample, an identifier 217 is shown to indicate that a determinedattribute of the building includes that it is an accessible floor plan(e.g., based on all of the first story being wheelchair accessible), aswell as identifiers 215 a-c corresponding to determined room types, andother identifiers 216 (e.g., 216 a) corresponding to other suchfeatures. While GUI controls and tags/labels and other identifiers arenot shown in the secondary pane 208 in this example, such informationmay be included in at least some embodiments. Also, as discussed withrespect to the GUI information elsewhere herein, the information shownin the illustrated panes of the GUI may be coordinated, such as toupdate the position and orientation information overlaid on the computermodel of the building to reflect a current position of the presenterdevice that recorded video shown in the secondary pane 208, to updatethe portion of the building shown in the interactive tour of secondarypane 209 to correspond to that similar position of the presenter device,to display and/or highlight particular overlaid information (e.g., roomtype tag 215 a) corresponding to a current room to which the videocurrently corresponds, etc. It will be appreciated that variousadditional types of information may be displayed in a similar manner inother embodiments and situations.

FIG. 2X continues the examples of FIGS. 2A-2W, and illustratesinformation similar to that of FIG. 2V and FIG. 2B, but in which one ormore participants in the virtual showing tour have submitted questionsand/or requests related to points of interest (POIs) in or around thebuilding (e.g., in one or more external areas on the property containingthe building), such as by indicating locations in visual data from thebuilding (e.g., images, a video, etc.) and/or on associated buildinginformation (e.g., a 3D computer model or other floorplan), or byotherwise describing one or more POIs, and optionally providing textand/or audio related to the questions or requests. Such POI indicationsfrom the one or more participants may occur, for example, during avirtual tour (e.g., by the participant selecting or otherwise indicatingcurrent visual data being shown in a video for a virtual tour), or inadvance of a virtual tour (e.g., on images or other building informationdisplayed in a GUI on a client device of the participant), withnon-exclusive examples 234 a and 234 b shown in panes 208 and 209corresponding to windows in the living room. Such POI indications mayfurther be used to direct activities during the virtual tour, such as todirect a user of a presenter device at the building to obtain visualand/or audio data from or for a particular POI and/or to provide audiocommentary related to the POI, and/or to cause another participant inthe virtual tour to respond as part of an interactive communicationsession during the virtual tour, with one example including using suchPOI indications to direct all of the virtual tour (e.g., for the virtualtour to travel to locations of each of the POI indications and providecorresponding information about them) - in other embodiments, a userassociated with the building (whether or not a participant in thevirtual tour) may provide response information after the virtual tour tothe participant who submitted the question or other request. In theillustrated example of FIG. 2X, one of the virtual tour participants (inthis example referred to as ‘End User E’) has indicated the picturewindow in the living room as a POI 234 a (whether before or during thevirtual tour), and another participant in the virtual tour (whether theuser of the presenter device at the building or another one of the videorecipients) may provide a textual or audio response (e.g., as part ofdisplayed information 213 displayed in the GUI), as well as to directthe user of the presenter device to capture additional visual data inthe video of a particular type (in this example, to show the view outthe picture window from the living room), such as by the presenterdevice moving to the picture window and directing a camera outwards.Other non-exclusive examples of questions and/or requests includerequesting information about when an indicated wall was last painted,when a water heater was last replaced or serviced, etc. Non-exclusiveexamples of building locations at which additional visual data isrequested include near an indicated window, around the kitchen, insidean indicated closet, at one or more indicated locations outside thebuilding, etc. In addition, such POI indications may be determined inother manners in other embodiments, such as based on visual analysis ofimages and/or other captured building data (e.g., using one or moretrained machine learning models), supplied by a property owner and/orassociated property listing agent, etc., and whether in addition to orinstead of by the ‘End User E’ virtual tour participant in the exampleof FIG. 2X. Furthermore, such POI indications may be further used inother manners in some embodiments and situations, such as to search forother buildings that have similar POIs, whether in addition to orinstead of using the POI indications to direct activities during avirtual tour.

Various details have been provided with respect to FIGS. 2A-2X, but itwill be appreciated that the provided details are non-exclusive examplesincluded for illustrative purposes, and other embodiments may beperformed in other manners without some or all such details. Forexample, while not illustrated in the examples of FIGS. 2A-2X, the GUIcould have other forms in other embodiments, such as to have otheruser-selectable and/or user-modifiable controls (whether instead of orin addition to the illustrated controls), and/or to have particularcontrols be accessed and used in other manners, and/or to be displayedor otherwise presented on other types of devices (e.g., desktop orlaptop computers, tablet computers, etc.), and/or to be displayed orotherwise presented in other types of formats and layouts (e.g., to havemore or less GUI panes, to have different layouts and/or sizes of GUIpanes, to display multiple types of information simultaneously withoutusing GUI panes like those of the examples, etc.). In addition,additional types of functionality related to displaying or otherwisepresenting building interior information may be provided in otherembodiments. Various other changes to the GUI may be further made inother embodiments.

FIG. 3 is a block diagram illustrating an embodiment of one or moreserver computing systems 300 executing an implementation of a BIIPsystem 340, optionally one or more server computing systems 380executing an implementation of an ICA system 389, and optionally one ormore server computing systems 370 executing an implementation of an FMGMsystem 379 -the server computing system(s) and BIIP and/or FMGM and/orICA systems may be implemented using a plurality of hardware componentsthat form electronic circuits suitable for and configured to, when incombined operation, perform at least some of the techniques describedherein. In the illustrated embodiment, each server computing system 300includes one or more hardware central processing units (“CPUs”) or otherhardware processors 305, various input/output (“I/O”) components 310,storage 320, and memory 330, with the illustrated I/O componentsincluding a display 311, a network connection 312, a computer-readablemedia drive 313, and other I/O devices 315 (e.g., keyboards, mice orother pointing devices, microphones, speakers, GPS receivers, etc.).Each server computing system 380 and 370 may have similar components,although only one or more hardware processors 381 and 371, memory 387and 377, storage 385 and 375, and I/O components 382 and 372,respectively, are illustrated for the sake of brevity.

The server computing system(s) 300 and executing BIIP system 340, andserver computing system(s) 380 and executing ICA system 389 if present,and server computing system(s) 370 and executing FMGM system 379 ifpresent, may communicate with each other and with other computingsystems and devices in this illustrated embodiment via one or morenetworks 399 (e.g., the Internet, one or more cellular telephonenetworks, etc.), such as to interact with user client computing devices390 (e.g., used to view a GUI that simultaneously displays multipletypes of information about an interior of a building in an integratedand coordinated manner, or to otherwise present information about abuilding), and/or one or more mobile image acquisition devices 360(e.g., used to acquire images and optionally other information forbuildings or other environments to be modeled), and/or optionally othernavigable devices 395 that receive and use computer models and/or otherbuilding information (e.g., 2D floor maps) for navigation purposes(e.g., for use by semi-autonomous or fully autonomous vehicles or otherdevices). In other embodiments, some of the described functionality maybe combined in less computing systems, such as to combine the ICA system389 and the image acquisition functionality of device(s) 360 in a singlesystem or device (e.g., via the optional ICA application 368 executingin memory 367 of the mobile device 360), to combine the BIIP system 340and/or the ICA system 389 and/or the FMGM system 379 in a single systemor device, to combine the BIIP system 340 and the ICA system 389 and theFMGM system 379 and the image acquisition functionality of device(s) 360in a single system or device, etc.

In the illustrated example of FIG. 3 , an embodiment of the BIIP system340 executes in memory 330 of the server computing system(s) 300 inorder to perform at least some of the described techniques, such as byusing the processor(s) 305 to execute software instructions of thesystem 340 in a manner that configures the processor(s) 305 andcomputing system 300 to perform automated operations that implementthose described techniques. The illustrated embodiment of the BIIPsystem may include one or more components, not shown, to each performportions of the functionality of the BIIP system, and the memory mayfurther optionally execute one or more other programs 335 - as onespecific example, a copy of the ICA and/or FMGM systems may each executeas one of the other programs 335 in at least some embodiments, such asinstead of or in addition to the ICA system 389 on the server computingsystem(s) 380 and the FMGM system 379 on the server computing system(s)370. The BIIP system 340 may further, during its operation, store and/orretrieve various types of data on storage 320 (e.g., in one or moredatabases or other data structures), such as various types of userinformation 322, images and other media information 325 acquired from orabout a building (e.g., received from ICA system 389, to provide tousers of client computing devices 390 for display; etc.), generatedcomputer models and optionally floor maps and other associated mappinginformation 326 (e.g., received from FMGM system 379, such as generatedand saved 2.5D and/or 3D models, 2D floor maps, etc.), other types ofbuilding information 324 (e.g., annotations and other descriptions,information about points of interest, information about surroundingbuildings and/or vegetation and/or other exterior information for thebuilding, etc., such as to be determined in part or in whole fromautomated analysis of visual data of captured images; as well asinformation from end users, such as questions; information from BIIPsystem operator users or other authorized users, such as answers toquestions; etc.), and/or various types of optional additionalinformation 328 (e.g., various analytical information related toanalysis of other building information and/or end user activities).

In addition, an embodiment of the ICA system 389 executes in memory 387of the server computing system(s) 380 in the illustrated embodiment inorder to perform automated operations related to acquiring images ofbuilding interiors (and optionally exteriors of buildings, includingtheir surroundings), such as by using the processor(s) 381 to executesoftware instructions of the system 389 in a manner that configures theprocessor(s) 381 and computing system 380 to perform such automatedoperations. Similarly, an embodiment of the FMGM system 379 executes inmemory 377 of the server computing system(s) 370 in the illustratedembodiment in order to perform automated operations related togenerating computer models and optionally floor maps of buildinginteriors, such as by using the processor(s) 371 to execute softwareinstructions of the system 379 in a manner that configures theprocessor(s) 371 and computing system 370 to perform such automatedoperations. The illustrated embodiments of the ICA and/or FMGM systemsmay each include one or more components, not shown, to each performportions of the functionality of their respective ICA or FMGM system,and the respective computer memories may further optionally execute oneor more other programs (not shown). The ICA system 389 and/or FMGMsystem 379 may further, during their operation, store and/or retrievevarious types of data on storage 385 or 375, respectively (e.g., in oneor more databases or other data structures), such as acquired images386, generated computer models 376 (e.g., generated and saved 2.5Dand/or 3D models) and optionally floor maps and other associatedinformation 376 (e.g., building and room dimensions for use withassociated floor plans, additional images and/or annotation information,various analytical information related to presentation or other use ofone or more building interiors or other environments, etc.) - while notillustrated in FIG. 3 , the ICA and/or FMGM systems may further storeand use additional types of information, such as about system operatorusers of the respective systems, metadata about acquisition of images tobe analyzed, etc.

Some or all of the user client computing devices 390 (e.g., mobiledevices), mobile image acquisition devices 360, optional other navigabledevices 395 and other computing systems (not shown) may similarlyinclude some or all of the same types of components illustrated forserver computing system 300. As one non-limiting example, the mobileimage acquisition device(s) 360 are each shown to include one or morehardware CPU(s) 361, I/O components 362, storage 365, and memory 367,with one or both of a browser and one or more client applications 368(e.g., an application specific to the FMGM system and/or ICA systemand/or BIIP system) executing within memory 367, such as to participatein communication with the BIIP system 340, ICA system 389, FMGM system379 and/or other computing systems - the devices 360 each furtherinclude one or more imaging systems 364 and IMU hardware sensors 369,such as for use in acquisition of images and associated movement/traveldata of the device 360. While particular components are not illustratedfor the other navigable devices 395 or other computing systems 390, itwill be appreciated that they may include similar and/or additionalcomponents.

It will also be appreciated that computing systems 300, 370 and 380 andthe other systems and devices included within FIG. 3 are merelyillustrative and are not intended to limit the scope of the presentinvention. The systems and/or devices may instead each include multipleinteracting computing systems or devices, and may be connected to otherdevices that are not specifically illustrated, including via Bluetoothcommunication or other direct communication, through one or morenetworks such as the Internet, via the Web, or via one or more privatenetworks (e.g., mobile communication networks, etc.). More generally, adevice or other computing system may comprise any combination ofhardware that may interact and perform the described types offunctionality, optionally when programmed or otherwise configured withparticular software instructions and/or data structures, includingwithout limitation desktop or other computers (e.g., tablets, slates,etc.), database servers, network storage devices and other networkdevices, smart phones and other cell phones, consumer electronics,wearable devices, digital music player devices, handheld gaming devices,PDAs, wireless phones, Internet appliances, and various other consumerproducts that include appropriate communication capabilities. Inaddition, the functionality provided by the illustrated systems 340and/or 379 and/or 389 may in some embodiments each be distributed invarious components, some of the described functionality of the systems340 and/or 379 and/or 389 may not be provided, and/or other additionalfunctionality may be provided.

It will also be appreciated that, while various items are illustrated asbeing stored in memory or on storage while being used, these items orportions of them may be transferred between memory and other storagedevices for purposes of memory management and data integrity.Alternatively, in other embodiments some or all of the softwarecomponents and/or systems may execute in memory on another device andcommunicate with the illustrated computing systems via inter-computercommunication. Thus, in some embodiments, some or all of the describedtechniques may be performed by hardware means that include one or moreprocessors and/or memory and/or storage when configured by one or moresoftware programs (e.g., by the BIIP system software 340 executing onserver computing systems 300 and/or on devices 360, by the ICA systemsoftware 389 executing on server computing systems 380, by the FMGMsystem software 379 executing on server computing systems 370, etc.)and/or data structures, such as by execution of software instructions ofthe one or more software programs and/or by storage of such softwareinstructions and/or data structures, and such as to perform algorithmsas described in the flow charts and other disclosure herein.Furthermore, in some embodiments, some or all of the systems and/orcomponents may be implemented or provided in other manners, such as byconsisting of one or more means that are implemented partially or fullyin firmware and/or hardware (e.g., rather than as a means implemented inwhole or in part by software instructions that configure a particularCPU or other processor), including, but not limited to, one or moreapplication-specific integrated circuits (ASICs), standard integratedcircuits, controllers (e.g., by executing appropriate instructions, andincluding microcontrollers and/or embedded controllers),field-programmable gate arrays (FPGAs), complex programmable logicdevices (CPLDs), etc. Some or all of the components, systems and datastructures may also be stored (e.g., as software instructions orstructured data) on a non-transitory computer-readable storage mediums,such as a hard disk or flash drive or other non-volatile storage device,volatile or non-volatile memory (e.g., RAM or flash RAM), a networkstorage device, or a portable media article (e.g., a DVD disk, a CDdisk, an optical disk, a flash memory device, etc.) to be read by anappropriate drive or via an appropriate connection. The systems,components and data structures may also in some embodiments betransmitted via generated data signals (e.g., as part of a carrier waveor other analog or digital propagated signal) on a variety ofcomputer-readable transmission mediums, including wireless-based andwired/cable-based mediums, and may take a variety of forms (e.g., aspart of a single or multiplexed analog signal, or as multiple discretedigital packets or frames). Such computer program products may also takeother forms in other embodiments. Accordingly, embodiments of thepresent disclosure may be practiced with other computer systemconfigurations.

FIG. 4 illustrates an example flow diagram of an embodiment of an ICASystem routine 400. The routine may be performed by, for example, theICA system 160 of FIG. 1 , the ICA system 389 of FIG. 3 , and/or the ICAsystem described with respect to FIGS. 1-2X and as otherwise describedherein, such as to acquire images (e.g., 360° spherical panorama images)at viewing locations within buildings or other structures, such as foruse in subsequent generation of related floor maps and/or other mappinginformation. While portions of the example routine 400 are discussedwith respect to acquiring particular types of images at particularviewing locations, it will be appreciated that this or a similar routinemay be used to acquire video or other data (e.g., audio), whetherinstead of or in addition to such images. In addition, while theillustrated embodiment acquires and uses information from the interiorof a target building, it will be appreciated that other embodiments mayperform similar techniques for other types of data, including fornon-building structures and/or for information external to one or moretarget buildings of interest. Furthermore, some or all of the routinemay be executed on a mobile device used by a user to acquire imageinformation, and/or by a system remote from such a mobile device.

The illustrated embodiment of the routine begins at block 405, whereinstructions or information are received. At block 410, the routinedetermines whether the received instructions or information indicate toacquire data representing a building interior, and if not continues toblock 490. Otherwise, the routine proceeds to block 412 to receive anindication from a user of a mobile image acquisition device to begin theimage acquisition process at a first viewing location. After block 412,the routine proceeds to block 415 in order to perform viewing locationimage acquisition activities in order to acquire an image (e.g., a 360°panorama image) for the viewing location in the interior of the targetbuilding of interest, optionally via one or more fisheye lenses on themobile device to provide horizontal coverage of at least 360° around avertical axis. As one non-exclusive example, the mobile imageacquisition device may be a rotating (scanning) panorama camera equippedwith a fisheye lens, such as a 180° fisheye giving a full sphere at 360°rotation. The routine may also optionally obtain annotation and/or otherinformation (e.g., information about points of interest, otherdescriptions, etc.) from the user regarding the viewing location and/orthe surrounding environment, such as for later use in presentation ofinformation regarding that viewing location and/or surroundingenvironment.

After block 415 is completed, the routine continues to block 420 todetermine if there are more viewing locations at which to acquireimages, such as based on corresponding information provided by the userof the mobile device, and/or based on automated tracking of viewinglocations visited (e.g., based on a predefined group of viewinglocations, based on acquiring at least one viewing location in each of apredefined group of rooms, etc.). If so, and when the user is ready tocontinue the process (if the device is carried by the user or itsmovement is otherwise controlled by the user), the routine continues toblock 422 to optionally initiate the capture of linking information(e.g., acceleration data and/or other data that changes as the devicemoves) during movement of the mobile device along a travel path awayfrom the current viewing location and towards a next viewing locationwithin the building interior. As described elsewhere herein, thecaptured linking information may include additional sensor data (e.g.,from one or more IMUs, or inertial measurement units, on the mobiledevice or otherwise carried by the user), as well as additional videoinformation, recorded during such movement. Initiating the capture ofsuch linking information may be performed in response to an explicitindication from a user of the mobile device or based on one or moreautomated analyses of information recorded from the mobile device. Inaddition, the routine may further optionally monitor the motion of themobile device in some embodiments during movement to the next viewinglocation, and provide one or more guidance cues to the user regardingthe motion of the mobile device, quality of the sensor data and/or videoinformation being captured, associated lighting/environmentalconditions, advisability of capturing a next viewing location,information about how to reach a defined next viewing location and/orabout when the defined next viewing location is reached, and any othersuitable aspects of capturing the linking information. Similarly, theroutine may optionally obtain annotation and/or other information (e.g.,information about points of interest, other descriptions, etc.) from theuser regarding the travel path, such as for later use in presentation ofinformation regarding that travel path or a resulting inter-panoramaconnection link. In block 424, the routine determines that the mobiledevice has arrived at the next viewing location (e.g., based on anindication from the user, based on the forward movement of the userstopping for at least a predefined amount of time, based on reaching apredefined spot for the next viewing location, etc.), for use as the newcurrent viewing location, and returns to block 415 in order to performthe viewing location image acquisition activities for the new currentviewing location.

If it is instead determined in block 420 that there are not any moreviewing locations at which to acquire image information for the currentbuilding or other structure, the routine proceeds to block 425 tooptionally analyze the viewing location information for the building orother structure, such as to identify possible additional coverage(and/or other information) to acquire within the building interior. Forexample, the ICA system may provide one or more notifications to theuser regarding the information acquired during capture of the multipleviewing locations and optionally corresponding linking information, suchas if it determines that one or more segments of the recordedinformation are of insufficient or undesirable quality, or do not appearto provide complete coverage of the building. After block 425, theroutine continues to block 435 to optionally preprocess the acquiredimages before their subsequent use for generating related mappinginformation, such as to perform an equirectangular projection for eachsuch image so that straight vertical data (e.g., the sides of a typicalrectangular door frame, a typical border between 2 adjacent walls, etc.)remains straight and in which straight horizontal data (e.g., the top ofa typical rectangular door frame, border between a wall and a floor,etc.) at a horizontal midline of the image remains straight but isincreasingly curved in the equirectangular projection image in a convexmanner relative to the horizontal midline as the distance increases inthe image from the horizontal midline. In at least some embodiments, anautomated analysis of some or all of the images may further be performedto determine corresponding information about the building interior, suchas types of rooms, points of interest in particular locations, etc. Inblock 477, the images and any associated generated or obtainedinformation is stored for later use, and optionally provided to an FMGMroutine for further analysis. FIGS. 5A-5B illustrate one example of aroutine for generating a floor map representation of a building interiorfrom the generated panorama information.

If it is instead determined in block 410 that the instructions or otherinformation recited in block 405 are not to acquire images and otherdata representing a building interior, the routine continues instead toblock 490 to perform any other indicated operations as appropriate, suchas any housekeeping tasks, to configure parameters to be used in variousoperations of the system (e.g., based at least in part on informationspecified by a user of the system, such as a user of a mobile device whocaptures one or more building interiors, an operator user of the ICAsystem, etc.), to obtain and store other information about users of thesystem, to respond to requests for generated and stored information,etc.

Following blocks 477 or 490, the routine proceeds to block 495 todetermine whether to continue, such as until an explicit indication toterminate is received, or instead only if an explicit indication tocontinue is received. If it is determined to continue, the routinereturns to block 405 to await additional instructions or information,and if not proceeds to step 499 and ends.

FIGS. 5A-5B illustrate an example embodiment of a flow diagram for aFloor Map Generation Manager (FMGM) System routine 500. The routine maybe performed by, for example, execution of the FMGM system 160 of FIG. 1, the FMGM system 379 of FIG. 3 , and/or an FMGM system as discussedwith respect to FIGS. 1-2X and elsewhere as described herein, such as togenerate a computer model and optionally other mapping information for adefined area based at least in part on images of the area. In theexample of FIGS. 5A-5B, the generated mapping information includes a 3Dcomputer model and 2D floor map of a building, such as a house, but inother embodiments, other types of mapping information may be determinedand generated for other types of buildings and used in other manners, asdiscussed elsewhere herein.

The illustrated embodiment of the routine begins at block 505, whereinformation or instructions are received. The routine continues to block510 to determine whether the instructions received in block 505 indicateto generate mapping information for an indicated building, optionallyalong with associated information about the building, and if so theroutine continues to perform blocks 515-588 to do so, and otherwisecontinues to block 590.

In block 515, the routine determines whether image information isalready available for the building, or if such information instead needsto be acquired. If it is determined in block 515 that the informationneeds to be acquired, the routine continues to block 520 to acquire suchinformation, optionally waiting for one or more users or devices to movethroughout the building and acquire panoramas or other images atmultiple viewing locations in multiple rooms of the building, and tooptionally further analyze the images and/or metadata information abouttheir acquisition to interconnect the images, as discussed in greaterdetail elsewhere herein - FIG. 4 provides one example embodiment of anICA system routine for performing such image acquisition. If it isinstead determined in block 515 that it is not necessary to acquire theimages, the routine continues instead to block 530 to obtain existingpanoramas or other images from multiple viewing locations in multiplerooms of the building, optionally along with interconnection informationfor the images and acquisition of metadata information related tomovement between the viewing locations, such as may in some situationshave been supplied in block 505 along with the correspondinginstructions.

After blocks 520 or 530, the routine continues to block 535 tooptionally obtain additional information about the building, such asfrom activities performed during acquisition and optionally analysis ofthe images, and/or from one or more external sources (e.g., onlinedatabases, information provided by one or more end users, etc.) - suchadditional information may include, for example, exterior dimensionsand/or shape of the building, additional images and/or annotationinformation acquired corresponding to particular locations within thebuilding (optionally for locations different from viewing locations ofthe acquired panorama or other images), etc.

After block 535, the routine continues to block 550 to determine, foreach room inside the building with one or more viewing locations andassociated acquired images, a room shape of the room for data in theimage(s) taken inside the room, and optionally a position within theroom of its viewing location(s), such as in an automated manner. Inblock 555, the routine further uses visual data in the images and/or theacquisition metadata for them to determine, for each room in thebuilding, any connecting passages in or out of the room, such as in anautomated manner. It will be appreciated that, while blocks 550-555, areillustrated as separate operations in this example, in some embodimentsa single analysis of the images may be performed to acquire or determinemultiple types of information, such as those discussed with respect toblocks 550-555.

In block 570, the routine then determines estimated positions of theroom shapes to create an initial 2D floor map, such as by connectinginter-room passages in their respective rooms, by optionally positioningroom shapes around determined viewing location positions (e.g., if theviewing location positions are inter-connected), and by optionallyapplying one or more constraints or optimizations. Such a floor map mayinclude, for example, relative position and shape information for thevarious rooms without providing any actual dimension information for theindividual rooms or building as a whole, and may further includemultiple linked or associated sub-maps (e.g., to reflect differentstories, levels, sections, etc.) of the building.

After block 570, the routine optionally performs one or more steps575-580 to determine and associate additional information with the floormap. In block 575, the routine optionally estimates the dimensions ofsome or all of the rooms, such as from analysis of images and/or theiracquisition metadata or from overall dimension information obtained forthe exterior of the building, and associates the estimated dimensionswith the floor map - it will be appreciated that if sufficientlydetailed dimension information were available, a floor plan (e.g.,architectural drawings, blueprints, etc.) may be generated from thefloor map. After block 575, the routine continues to block 580 tooptionally associate further information with the floor map (e.g., withparticular rooms or other locations within the building), such asadditional images and/or annotation information. After block 580, theroutine continues to block 583, where it analyzes, for each room in thebuilding with one or more image viewing locations, the image(s) from theimage viewing location(s) to determine information about aspects of thebuilding interior that are visible in the image(s) (e.g., types of roomsand/or associated functionality; structural elements and other physicalcomponents of the building interior such as one or more of walls,windows, doors, stairs, fireplaces, kitchen islands, cabinets, counters,lighting fixtures, plumbing fixtures, built-in elements such as sinksand/or showers/baths and/or ovens and/or stoves and/or trash compactorsand/or water heaters and/or HVAC (heating, ventilation and/or cooling)equipment and/or dish washers, moveable appliances such as refrigeratorsand/or clothes washers and/or dryers and/or microwaves, types ofinfrastructure such as using wiring and/or piping, including electricaloutlets and/or cable outlets and/or telephone outlets and/or air ventsand/or plumbing fixtures and/or sprinklers and/or lighting fixturesand/or speakers and/or speaker outlet/wiring locations and/or securitycameras, curtains and/or other furnishings, floor coverings, materialsused for surfaces such as kitchen countertops and/or wallpaper and/orpaint and/or floors, furniture, fire extinguishers, etc. and optionallytheir locations and/or brands and/or types; other types of points ofinterest in particular locations; etc.), before continuing to block 585.In block 585, the routine further estimates heights of walls in some orall rooms, such as from analysis of images and optionally sizes of knownobjects in the images, as well as height information about a camera whenthe images were acquired, and further uses such information to generatea 3D computer model of the building, with the 3D model and the floor mapbeing associated with each other.

After block 585, the routine continues to block 588 to store thegenerated mapping information and optionally other generatedinformation, and to optionally further use the generated mappinginformation, such as to provide the generated 3D computer model and/or2D floor map for display on one or more client devices, provide thatgenerated information to one or more other devices for use in automatingnavigation of those devices and/or associated vehicles or otherentities, etc.

If it is instead determined in block 510 that the information orinstructions received in block 505 are not to generate mappinginformation for an indicated building, the routine continues instead toblock 590 to perform one or more other indicated operations asappropriate. Such other operations may include, for example, receivingand responding to requests for previously generated computer modelsand/or floor maps and/or other generated information (e.g., requests forsuch information for use by a BIIP system in generating and presenting aGUI that simultaneously displays multiple types of information about abuilding in an integrated and coordinated manner, requests for suchinformation for display on one or more client devices, requests for suchinformation to provide it to one or more other devices for use inautomated navigation, etc.), obtaining and storing information aboutbuildings for use in later operations (e.g., information aboutdimensions, numbers or types of rooms, total square footage, adjacent ornearby other buildings, adjacent or nearby vegetation, exterior images,etc.), etc.

After blocks 588 or 590, the routine continues to block 595 to determinewhether to continue, such as until an explicit indication to terminateis received. If it is determined to continue, the routine returns toblock 505 to wait for and receive additional instructions orinformation, and otherwise continues to block 599 and ends.

FIG. 6 illustrates an example embodiment of a flow diagram for aBuilding Information Integrated Presentation (BIIP) system routine 600.The routine may be performed by, for example, execution of the BIIPsystem 140 of FIG. 1 , the BIIP system 340 of FIG. 3 , and/or a BIIPsystem as discussed with respect to FIGS. 1-2X and/or as describedelsewhere herein, such as to generate and present a GUI that displaysmultiple types of information about a building, such as simultaneouslyin an integrated and coordinated manner. In the example of FIG. 6 , theGUI uses multiple panes to simultaneously present multiple types ofinformation in an integrated and coordinated manner, but in otherembodiments, other types of GUIs with other types of structures andlayouts may be used to present the same or similar types of information,as discussed elsewhere herein, including with respect to the examples ofFIGS. 2A-2X.

The illustrated embodiment of the routine begins in block 605, whereinformation or instructions are received. The routine continues to block610 to determine whether the instructions in block 605 are to presentintegrated information for an indicated building, such as in acorresponding GUI. If so, the routine continues to perform blocks615-650, and otherwise continues to block 690. In particular, if it isdetermined in block 610 that the instructions received in block 605 areto present integrated information for an indicated building, the routinecontinues to block 615 to obtain building information of multiple typesfor the indicated building, such as a 3D model of the building, imagesof the interior (and optionally, exterior) of the building, videos ofthe interior (and optionally, exterior) of the building, informationabout an interactive tour of a plurality of viewing/capture locationswithin the building interior (and optionally, exterior) at which imageand/or other information was captured, a 2-D floor map or other floorplan, audio and/or textual descriptions of particular locations or areas(e.g., rooms, points of interest, etc.), simulated and/or actuallighting information, information about surrounding buildings and/orvegetation and/or other exterior aspects (vehicle traffic, foot traffic,noises, etc.), surfaces and/or areas available for virtual staging orotherwise for adding virtual objects, information about types ofbuilding information to use as POIs (e.g., information from automatedanalysis of visual data of images captured for the building), etc.

After block 615, the routine continues to block 620 to determine aninitial GUI display for the building (e.g., to retrieve a preexistinginitial GUI to be used; to retrieve predefined information to add to theGUI; to generate the initial GUI, such as from one or more selectionsmade in the instructions received in block 605; etc.), such as a GUIwith three or more separate panes that simultaneously show at leastthree types of information about the building in a coordinated manner.In at least some embodiments, the at least three types of informationmay include one or more images and/or videos, at least some of the 3Dcomputer model and/or 2D floor map model, at least some of aninteractive tour, etc., such as with all of the information focused on acommon location or other area of the building or other common feature oraspect. The GUI may further include various user-selectable controls(e.g., user-modifiable controls in which the user may modify a value orselect from multiple options; other user-selectable controls in whichfunctionality is toggled on and off or otherwise invoked upon selectionwith respect to one or more types of overlaid information, such ascorresponding to POIs for one or more types of building information,including for types of building information determined from automatedanalysis of visual data of images; other user-selectable controls inwhich functionality is toggled on and off or otherwise invoked uponselection with respect to time-based controls to determine what types ofcorresponding building information is displayed, such as based on timeof acquisition of that information; etc.) for display to an end user,such as within one or more of the separate panes or instead in separateareas of the GUI (e.g., at the top, at the bottom, on the left and/orright edges, etc.) - in at least some embodiments and situations, someor all of the user-selectable controls may be selected contextuallybased on other information that is displayed in the GUI, such as to havea primary pane (e.g., that is larger than other secondary page) and tooverlay the content shown in the primary pane with one or moreuser-selectable controls that are specific to that type of content. Ifthe GUI is to be displayed on the same computer that is executing theroutine 600, the routine further generates a visual representation tosupply to one or more display devices or other output devices of thatcomputer, and otherwise generates a visual representation fortransmission to another client device on which the visual representationof the GUI will be displayed. After block 620, the routine continues toblock 625 to provide the generated GUI visual representation to thecomputer device on which it will be displayed, and then waits for anevent to trigger a change in the GUI (e.g., a user selection to modifythe displayed information, a change in the underlying displayedinformation, an amount of time having elapsed since a last user inputand/or a last display activity by the routine, etc.).

After block 625, the routine continues to block 630 to determine whetherthe event that has occurred with respect to block 625 indicates a changein the information to be displayed for the current building, and if socontinues to block 635 to determine an updated visual representation ofthe GUI to use as a new current GUI visual representation with thechanged information, and then returns to block 625 to initiate displayof the updated visual representation. The change may occur, for example,based on a user selection (e.g., of a user-selectable control in thedisplayed GUI) and/or a change in the underlying displayed information.In at least some embodiments and situations, the updated visualrepresentation again shows multiple types of information in multiple GUIpanes that all focus on a common area, location or other aspect of thebuilding (whether the same common area, location or other aspect, orinstead a different one) and that further include user-selectablecontrols corresponding to one or more panes of the GUI - in otherembodiments and situations, information displayed in the GUI at a giventime may include all or substantially all of the displayed area of theGUI, such as in a single pane and optionally being of a single type. Ifit is instead determined in block 630 that the event that occurred withrespect to block 625 did not cause a change in displayed information forthe current building, the routine continues to block 650 to determinewhether the event instead indicates to display information for a newbuilding, and if so the routine returns to block 615 to initiate adisplay of information for the new building in an updated version of theGUI.

If it is instead determined in block 610 that the instructions orinformation received in block 605 are not to present integratedinformation for an indicated building, the routine continues instead toblock 690 to perform one or more other indicated operations asappropriate. Such other operations may include, for example, receivingand responding to requests for previously generated GUI informationand/or for underlying information that may be used in a GUI (e.g.,information for an indicated building for display in a standalone manneror otherwise in a manner that is not part of an integrated displayshowing multiple types of related information simultaneously orotherwise concurrently; information for an indicated building thatincludes an array of images, such as thumbnail images, along with visualindications of additional types of media or other building information,such as a 3D model floor plan and/or a 2D floor map and/or aninteractive tour; etc.), obtaining and storing information aboutbuildings for use in later GUI presentation operations (e.g.,information from external sources, such as one or more databases;information from end users, such as questions about an indicatedbuilding and/or an answer to an existing question and/or images or othermedia captured in or around a building; etc.), performing automatedanalysis of available building information and/or activities of one ormore end users to determine patterns or otherwise identify additionalinformation for subsequent use, etc.

After block 690, or if it is instead determined in block 650 that theevent is not to display information for a new building, the routinecontinues to block 695 to determine whether to continue, such as untilan explicit indication to terminate is received. If it is determined tocontinue, the routine returns to block 605 to wait for and receiveadditional instructions or information, and otherwise continues to block699 and ends.

Aspects of the present disclosure are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of thepresent disclosure. It will be appreciated that each block of theflowchart illustrations and/or block diagrams, and combinations ofblocks in the flowchart illustrations and/or block diagrams, can beimplemented by computer readable program instructions. It will befurther appreciated that in some implementations the functionalityprovided by the routines discussed above may be provided in alternativeways, such as being split among more routines or consolidated into fewerroutines. Similarly, in some implementations illustrated routines mayprovide more or less functionality than is described, such as when otherillustrated routines instead lack or include such functionalityrespectively, or when the amount of functionality that is provided isaltered. In addition, while various operations may be illustrated asbeing performed in a particular manner (e.g., in serial or in parallel,or synchronous or asynchronous) and/or in a particular order, in otherimplementations the operations may be performed in other orders and inother manners. Any data structures discussed above may also bestructured in different manners, such as by having a single datastructure split into multiple data structures and/or by having multipledata structures consolidated into a single data structure. Similarly, insome implementations illustrated data structures may store more or lessinformation than is described, such as when other illustrated datastructures instead lack or include such information respectively, orwhen the amount or types of information that is stored is altered.

From the foregoing it will be appreciated that, although specificembodiments have been described herein for purposes of illustration,various modifications may be made without deviating from the spirit andscope of the invention. Accordingly, the invention is not limited exceptas by corresponding claims and the elements recited by those claims. Inaddition, while certain aspects of the invention may be presented incertain claim forms at certain times, the inventors contemplate thevarious aspects of the invention in any available claim form. Forexample, while only some aspects of the invention may be recited asbeing embodied in a computer-readable medium at particular times, otheraspects may likewise be so embodied.

What is claimed is:
 1. A computer-implemented method comprising:selecting, by one or more computing systems, multiple types ofinformation about an indicated building to simultaneously include in adisplayed graphical user interface (GUI), the multiple types ofinformation including a video having at least visual data from asuccession of multiple locations at the indicated building, and furtherincluding one or more additional types of information that include avisual representation of a floor map of the indicated building; andpresenting, by the one or more computing systems and in the displayedGUI, the video and the one or more additional types of information,including presenting the video in a first pane of the GUI in asubstantially real-time manner relative to capturing of the video by apresenter device at the indicated building that is moving through theindicated building and transmitting the video as it is captured, andincluding presenting the visual representation of the floor map in asecond pane of the GUI simultaneously with the video and in acoordinated manner in which the visual representation of the floor mapis repeatedly updated to reflect current visual data of the videoincluding to visually indicate the succession of multiple locations, thepresenting including: (a) receiving, by the one or more computingsystems, one or more current frames of the video from the presenterdevice for a current one of the multiple locations; (b) analyzing, bythe one or more computing systems, visual data of the one or morecurrent frames to identify at least one of a room type of a room of theindicated building that is visible in the visual data of the one or morecurrent frames, or a feature type for a structural element of theindicated building that is visible in the visual data of the one or morecurrent frames; (c) displaying, by the one or more computing systems,the visual data of the one or more current frames in the first pane, anda current version of the visual representation of the floor map in thesecond pane concurrently with the video, wherein the current version ofthe visual representation of the floor map includes at least a portionof the floor map of the indicated building showing a position of thecurrent one location on the at least portion of the floor map andshowing at least two-dimensional structural information for at leastsome of the indicated building; (d) overlaying, by the one or morecomputing systems and on at least one of the first or second panes, adetermined textual identifier for the identified at least one of theroom type or the feature type; and (e) repeating elements (a)-(d) untilpresentation of the video is ended.
 2. The computer-implemented methodof claim 1 wherein the presenting of the video and the one or moreadditional types of information includes transmitting the video and theone or more additional types of information to each of multiple clientdevices to cause concurrent display of the video and the one or moreadditional types of information on each of the multiple client devicesas part of a shared virtual showing tour of the building.
 3. Thecomputer-implemented method of claim 2 further comprising maintaining,during the presenting of the video, an interactive communication sessionbetween the multiple client devices, wherein the interactivecommunication session includes transmitting at least one of first audiodata or first textual data between all of the multiple client devices,and further includes transmitting at least one of second audio data orsecond textual data between only a selected pair of the multiple clientdevices.
 4. The computer-implemented method of claim 1 wherein eachanalyzing of the visual data of the one or more current frames includesusing machine learning to identify a room type of a room of theindicated building that is visible in the visual data of those one ormore current frames, and each overlaying of the determined textualidentifier includes displaying a determined textual identifier for thatidentified room type in the first pane.
 5. The computer-implementedmethod of claim 1 wherein the one or more additional types ofinformation further includes additional visual data and user-selectablecontrols for an interactive tour of a plurality of viewing locations atthe indicated building using images captured at each of the plurality ofviewing locations, and wherein the presenting of the video and the oneor more additional types of information includes presenting theadditional visual data and the user-selectable controls in a third paneof the displayed GUI concurrently with the video, and modifyinginformation shown in the third pane for the interactive tour in responseto one or more selections of one or more of the user-selectablecontrols.
 6. The computer-implemented method of claim 1 wherein thepresenting of the video and the one or more additional types ofinformation includes displaying user-selectable controls related tooverlaying one or more further types of information on at least one ofthe video or the one or more additional types of information, andmodifying information displayed in at least one of the first or secondpanes in response to one or more selections of one or more of theuser-selectable controls, the modifying including adding and/or removingthe overlaying of the one or more further types of information.
 7. Asystem comprising: one or more hardware processors of one or morecomputing systems; and one or more memories with stored instructionsthat, when executed by at least one of the one or more hardwareprocessors, cause the one or more computing systems to perform automatedoperations including at least: selecting multiple pieces of informationabout an indicated building to include concurrently in a displayedgraphical user interface (GUI), wherein the multiple pieces ofinformation include a video having at least visual data from one or morelocations at the indicated building, and further include one or moreadditional pieces of information about the indicated building, the oneor more additional pieces of information including at least one of avisual representation of a floor map of the indicated building, or oneor more images captured at the indicated building, or visual data anduser-selectable controls for an interactive virtual tour of theindicated building; and providing, via the displayed GUI, the videoconcurrently with the one or more additional pieces of information in acoordinated manner in which the one or more additional pieces ofinformation are updated to reflect current visual data of the video,including repeatedly providing a portion of the video for a current oneof the one or more locations and providing a portion of the one or moreadditional pieces of information that reflects the current one location.8. The system of claim 7 wherein the video includes visual data formultiple locations at the indicated building as a presenter device atthe indicated building moves through the indicated building andtransmits the video as it is captured, and wherein the providing of thevideo via the displayed GUI is performed in a substantially real-timemanner relative to transmitting of the video by the presenter device. 9.The system of claim 7 wherein the providing of the video and the one ormore additional pieces of information includes using multiple panes inthe displayed GUI, including to display the video in a first pane and tosimultaneously display the one or more additional pieces of informationin one or more second panes.
 10. The system of claim 7 wherein the oneor more additional pieces of information include at least a portion of afloor map of the indicated building that shows structural informationfor at least some of the indicated building and is repeatedly updated toreflect the current one location for the video.
 11. The system of claim7 wherein the one or more computing systems include a client device onwhich the GUI is displayed, and wherein the providing of the video andthe one or more additional pieces of information includes presenting thevideo and the one or more additional pieces of information in thedisplayed GUI on the client device.
 12. The system of claim 7 whereinthe one or more computing systems include at least one server computingsystem, and wherein the providing of the video and the one or moreadditional pieces of information includes transmitting the video and theone or more additional pieces of information over one or more computernetworks to at least one client device for display in a GUI on the atleast one client device.
 13. The system of claim 7 wherein the one ormore additional pieces of information includes visual data anduser-selectable controls for an interactive tour of a plurality ofviewing locations at the indicated building, and wherein providing ofthe one or more additional pieces of information includes modifyinginformation shown for the interactive tour in response to one or moreselections of one or more of the user-selectable controls.
 14. Thesystem of claim 7 wherein the providing of the video and the one or moreadditional pieces of information includes providing user-selectablecontrols related to overlaying one or more further pieces of informationon at least one of the video or the one or more additional pieces ofinformation, and modifying displayed information in response to one ormore selections of one or more of the user-selectable controls to addand/or remove the overlaying of the one or more further pieces ofinformation.
 15. The system of claim 7 wherein the automated operationsfurther include analyzing the visual data of the video to identify atleast one of a room type of a room of the indicated building that isvisible in the visual data, or a feature type for a structural elementof the indicated building that is visible in the visual data, andwherein the providing of the video and the one or more additional piecesof information includes displaying a determined textual identifier forthe identified at least one of the room type or the feature type with atleast one of the video or the one or more additional pieces ofinformation.
 16. The system of claim 7 wherein the video includes visualdata for multiple locations at the indicated building captured by apresenter device at the indicated building as the presenter device movesthrough the indicated building to provide a tour of the indicatedbuilding, and wherein the providing of the video and the one or moreadditional pieces of information is performed for each of multipleclient devices to cause concurrent display of the video and the one ormore additional pieces of information on each of the multiple clientdevices.
 17. The system of claim 16 wherein the automated operationsfurther include maintaining, during the providing of the video, aninteractive communication session between the multiple client devices,wherein the interactive communication session includes transmitting atleast one of first audio data or first textual data between the multipleclient devices.
 18. The system of claim 7 wherein the providing of thevideo concurrently with the one or more additional pieces of informationincludes causing presentation of the video and the one or moreadditional pieces of information on one or more client devices as partof a virtual tour of the indicated building, and wherein the automatedoperations further include: recording, in response to a request sentfrom at least one of the one or more client devices, at least the videofor the virtual tour; and providing, at a time after completion of thevideo, the recorded at least video to a recipient device to causepresentation of the recorded at least video on the recipient device. 19.The system of claim 7 wherein the providing of the video concurrentlywith the one or more additional pieces of information includes causingpresentation of the video and the one or more additional pieces ofinformation on one or more client devices as part of a virtual tour ofthe indicated building, and wherein the automated operations furtherinclude, during the providing of the video concurrently with the one ormore additional pieces of information: receiving a communication sentfrom one of the one or more client devices that identifies a point ofinterest at the indicated building, the point of interest beingindicated on the one client device via a selection in a GUI on the oneclient device in which the video and the one or more additional piecesof information are being presented; and providing a response to the oneclient device with information about the point of the interest that isreceived from at least one user associated with the indicated building.20. The system of claim 19 wherein the presentation of the video and theone or more additional pieces of information further occurs on multipleclient devices, wherein the response is received during the virtual tourfrom an additional one of the multiple client devices separate from theone client device, and wherein the providing of the response furtheroccurs during the virtual tour via at least one of a communication thatoccurs as part of the video, or a communication between the additionalone client device and the one client device.
 21. The system of claim 7wherein the automated operations further include, before the providingof the video concurrently with the one or more additional pieces ofinformation, receiving one or more indications from one or more users ofone or more points of interest at the indicated building, wherein theautomated operations further include identifying the one or more pointsof interest to a user associated with a presenter device, wherein thevideo is transmitted from the presenter device at the indicated buildingand includes at least visual data about the one or more points ofinterest in response to the identifying, and wherein the providing ofthe video concurrently with the one or more additional pieces ofinformation includes causing presentation of the video and the one ormore additional pieces of information to the one or more users of one ormore client devices.
 22. The system of claim 7 wherein the automatedoperations further include receiving a communication sent from a firstclient device that identifies multiple points of interest at theindicated building, the multiple points of interest being indicated onthe first client device via a selection in a GUI on the first clientdevice in information about the indicated building is being presented,and wherein the providing of the video concurrently with the one or moreadditional pieces of information includes causing presentation of thevideo and the one or more additional pieces of information on one ormore client devices that include the first client device as part of avirtual tour of the indicated building, the virtual tour including apresenter device at the indicated building moving to positions of eachof the multiple points of interest and providing at least visual datafor the video related to that point of interest.
 23. The system of claim7 wherein the automated operations further include: receiving acommunication sent from a client device that identifies one or morepoints of interest at the indicated building, the one or more points ofinterest being indicated on the client device via a selection in a GUIon the client device in which information for the indicated building isbeing presented; performing a search of other buildings based at leastin part on the identified one or more points of interest to generatesearch results that include at least one other building matching theidentified one or more points of interest; and transmitting, to theclient device, information about the at least one other building inresponse to the received communication.
 24. A non-transitorycomputer-readable medium having stored contents that cause one or morecomputing systems to perform automated operations including at least:selecting, by the one or more computing systems, multiple pieces ofinformation about an indicated building to include concurrently in adisplayed graphical user interface (GUI), wherein the multiple pieces ofinformation include a video having at least visual data from one or morelocations at the indicated building, and further include one or moreadditional pieces of information about the indicated building, the oneor more additional pieces of information including at least one or moretag identifiers for one or more rooms visible in the visual data; andproviding, by the one or more computing systems and via the displayedGUI, the video concurrently with the one or more additional pieces ofinformation in a coordinated manner in which the one or more additionalpieces of information is updated to reflect current visual data of thevideo, including repeatedly providing a portion of the video for acurrent one of the one or more locations and concurrently providing atleast one of the one or more tag identifiers that is determined for aroom of the indicated building visible in the visual data of the portionof the video and automatically identified from analysis of the visualdata of the portion of the video.
 25. The non-transitorycomputer-readable medium of claim 24 wherein the one or more additionalpieces of information further include one or more additional tagidentifiers for one or more features included in at least one of the oneor more rooms, and wherein the providing of the video concurrently withthe one or more additional pieces of information further includes, whileproviding at least one portion of the video, concurrently providing atleast one of the one or more additional tag identifiers that isdetermined for at least one of the one or more features visible in thevisual data of the at least one portion of the video and automaticallyidentified from analysis of the visual data of the at least one portionof the video.
 26. The non-transitory computer-readable medium of claim24 wherein the one or more tag identifiers are in at least one oftextual form or audio form, and wherein providing of each of the one ormore tag identifiers includes visually displaying that tag identifier ifit is in textual form and includes audibly playing that tag identifierif it is in audio form.
 27. The non-transitory computer-readable mediumof claim 24 wherein the video includes visual data for multiplelocations at the indicated building as a presenter device at theindicated building moves through the indicated building and transmitsthe video as it is captured, and wherein the providing of the video viathe displayed GUI is performed in a substantially real-time mannerrelative to transmitting of the video by the presenter device.
 28. Thenon-transitory computer-readable medium of claim 24 wherein theproviding of the video and the one or more additional pieces ofinformation includes using multiple panes in the displayed GUI,including to display the video in a first pane and to simultaneouslydisplay the one or more additional pieces of information in one or moresecond panes, and wherein the one or more additional pieces ofinformation include at least a portion of a floor map of the indicatedbuilding that shows structural information for at least some of theindicated building and is updated to reflect the current one locationfor the video.
 29. The non-transitory computer-readable medium of claim24 wherein the one or more computing systems include at least one servercomputing system, and wherein the providing of the video and the one ormore additional pieces of information includes transmitting the videoand the one or more additional pieces of information over one or morecomputer networks to a client device for display in the GUI on theclient device.
 30. The non-transitory computer-readable medium of claim24 wherein the providing of the video and the one or more additionalpieces of information includes providing user-selectable controlsrelated to one or more further pieces of information, and modifyingdisplayed information in response to one or more selections of one ormore of the user-selectable controls to add and/or remove at least oneof the one or more further pieces of information.
 31. The non-transitorycomputer-readable medium of claim 24 wherein the video includes visualdata for multiple locations at the indicated building captured by apresenter device at the indicated building as the presenter device movesthrough the indicated building to provide a tour of the indicatedbuilding, and wherein the providing of the video and the one or moreadditional pieces of information is performed for each of multipleclient devices to cause concurrent display of the video and the one ormore additional pieces of information on each of the multiple clientdevices.
 32. The non-transitory computer-readable medium of claim 31wherein the automated operations further include maintaining, during theproviding of the video, an interactive communication session between themultiple client devices, wherein the interactive communication sessionincludes transmitting at least one of first audio data or first textualdata between the multiple client devices.